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Detachment  1,  Air  Force  Operational  Test  and  Evaluation  Center  will  conduct  a 
Qualification  Operational  Test  and  Evaluation  (QOT&E)  on  the  SEON 
Upgrade/Replacement  Program  during  the  fourth  quarter  of  FY  1993  and  the  first 
quarter  of  FY  1994  at  the  Palehua  Solar  Observatory  in  Hawaii.  The  purpose  of  tf 
QOT&E  is  to  determine  the  operational  suitability  and  effectiveness  of  the 
upgraded  SEON  so  decision  makers  can  decide  what  changes,  if  any,  need  to  be  mad*: 
prior  to  upgrading  the  other  sites.  SEON  measures  solar  events  to  predict  theix 
impact  on  manned  space  activities  and  satellite  communication,  and  validates  ionc 
spheric  disturbances.  SEON  interfaces  with  the  Air  Force  Space  Forecast  Center 
the  Air  Force  Global  Weather  Central.  It  is  comprised  of  five  Solui  Observing 
Optical  Networks  ( SOONs )  and  four  Radio  Solar  Telescope  Networks  ( RSTNs ) 
longitudinally  separated  to  ensure  the  sun  is  constantly  monitored.  Each  SOON  i: 
composed  of  a  solar  optical  telescope  (AN/FMQ-7 )  with  automatic  solar  trackina . 
Each  PSTN  is  composed  of  a  solar  radio  telescope  ( AN/FP.P-95  >  with  automatic  r.olai 
tracking.  This  equipment  also  measures  the  magnitude  of  solar  flares  and  prepare 
the  necessary  solar  event  messages  for  release  by  the  operator  into  the  Automater 
»at  her  tletwor  k  . 


14  Subjh  '  r:*v , 

■Tolar  Electro-Optical  Network,  NEON, 
ANFMQ-7,  AN  FPP  c  . 


y.OOUi  r-CTU. 


15  NUMBER  OF  PAGES 


16  °RICE  CODE 


)0  UMITATlON  OF  ABSTRACT 


b.  C'OI-2:  Do  the  SEON  computers  provide  sufficient  capabilities  to  support  the  mission 
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d.  COM: 
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g.  COI-7: 

h.  COI-8: 
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Does  the  upgraded  RSTN  allow  detection  of  solar  events  ' 

Does  the  upgraded  SOON  allow  detection  ot  solar  events  ' 

Is  the  SEON  software  supportable? 

Does  the  SEON  adequately  report  solar  events? 

Is  the  SEON  logistically  supportable? 

Does  the  SEON  facility  provide  adequate  power  and  environmental  support  .' 
Is  the  human  factors  engineering  of  the  SEON  adequate? 


5.  This  QOT&E  will  be  managed  by  Det  1,  AFOTEC  and  supported  by  Detachment  5.  Air  Force 
Space  Forecast  Center.  Palehua,  Hawaii,  and  the  15th  Communications  Squadron,  Hickam  AFB. 
Hawaii.  The  test  will  be  representative  of  the  operational  scenarios  users  are  expected  to  employ 
during  actual  operations.  All  key  user  operational  issues  will  be  addressed  during  this  QOT&E. 
The  test  will  be  conducted  over  a  90-day  period,  and  will  test  the  upgraded  equipment  and 
software  and  their  effect  on  the  mission  capability  of  the  SEON.  Testing  will  be  conducted 
during  normal  SEON  operational  duty  hours,  which  are  sunup  to  sundown  (15  hours)  each  day 
for  a  total  1 350  hours  .  The  90  days  will  be  broken  down  into  three  30-day  phases:  Simulations. 
Normal  Operations,  and  Software.  Phase  1.  Simulations,  conducted  during  the  first  30  days,  w  ill 
consist  of  initial  system  start-up.  solar  event  simulations,  software  and  hardware  maintainability, 
and  a  safety  evaluation.  Phase  2,  Normal  Operations,  conducted  during  the  middle  30  days,  will 
evaluate  the  ability  to  perform  the  mission  using  real-time  solar  data.  This  phase  also  includes 
normal  maintenance  and  the  evaluation  of  the  uninterruptable  power  source.  Phase  3.  Software, 
will  evaluate  the  on-site  programmers  ability  to  perform  depot  maintenance  on  the  system 
application  software.  This  phase  also  includes  human  factors  engineering,  and  software  usability. 
There  are  no  known  test  limitations  identifiable  before  test  start.  The  user.  Air  Weather  Service, 
expects  a  representative  sample  of  solar  events  to  occur  during  the  test. 


6.  The  test  directors  will  submit  a  report  that  addresses  the  using  commands'  critical  operational 
issues.  Briefings  in  support  of  the  final  report  will  be  provided,  as  required.  Status  reports  will 
be  sent  bi-weekly  during  the  QOT&E.  Activity  and  Significant  Event  reports  will  be  prepared, 
as  required. 
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EXECUTIVE  SI  MM  ARY 


1.  Detachment  1.  Air  Force  Operational  Test  and  Evaluation  (.'enter  (Det  1.  AFOTEOi  will 
conduct  a  Qualification  Operational  Test  and  Evaluation  (QOT&E)  on  the  Solar  Electro-Optical 
Network  (SEON)  Upgrade/Replacemcnt  Program  during  the  fourth  quarter  of  I  V  1 003  and  the 
first  quarter  of  FY  1994  at  the  Palehua  Solar  Observatory  in  Hawaii.  The  purpose  of  this 
QOT&E  is  to  determine  the  operational  suitability  and  effectiveness  of  the  upgraded  SEON  prior 
to  considering  upgrading  the  other  sites. 

2.  The  SEON  is  the  only  current  or  planned  ne'work  that  can  provide  optical  and  radio  solar 
observations  support  to  the  Department  of  Defense  (DOD).  It  measures  solar  events  to  predict 
their  impact  on  manned  space  activities  and  satellite  communications,  and  validates  ionospheric 
disturbances.  The  SEON  interfaces  with  the  Air  Force  Space  Forecast  Center  and  the  Air  Force 
Global  Weather  Central,  and  is  required  to  alert  these  facilities  of  all  significant  solar  events 
w'ithin  2  minutes  of  their  occurrence.  The  SEON  is  comprised  of  five  Solar  Observing  Optical 
Networks  (SOONs)  and  four  Radio  Solar  Telescope  Networks  (RSTNs)  longitudinally  separated 
to  ensure  the  sun  is  constantly  monitored.  Each  SOON  is  composed  of  a  solar  optical  telescope 
(AN/FMQ-7)  with  automatic  solar  tracking.  Each  RSTN  is  composed  of  a  solar  radio  telescope 
(AN/FRR-95)  with  automatic  solar  tracking.  The  data  from  each  solar  observatory  is  collected 
and  analyzed  using  automated  computer  processing  equipment.  This  equipment  also  measures 
the  magnitude  of  solar  flares  and  prepares  the  necessary  solar  event  messages  for  release  by  the 
operator  into  the  Automated  Weather  Network.  The  sites  at  Palehua.  Hawaii,  Learmonth, 
Australia,  and  San  Vito  Air  Station,  Italy,  have  both  the  SOON  and  RSTN  and  are  referred  to 
as  combined  sites.  The  sites  at  Holloman  AFB,  New  Mexico  and  Ramey,  Puerto  Rico,  only  have 
a  SOON.  The  site  at  Sagamore  Hill,  Massachusetts,  only  has  a  RSTN.  The  upgraded  system 
will  consist  of  new  charge-coupled  device  (CCD)  cameras  in  the  SOON,  replacement  of  analog- 
to-digital  and  digital-to-analog  converters,  relay  drivers,  and  logic  cards  in  the  SOON  with  an 
Intelligent  Data  Acquisition  System  (IDAS),  replacement  of  the  sweep  frequency  interferometric 
radiometer  (SFIR)  in  the  RSTN,  and  replacement  of  the  computer  hardware  and  software  to 
operate  the  SOON  and  RSTN. 

3.  This  program  was  initiated  due  to  the  decreasing  maintenance  supportability  and  limited 
operational  effectiveness  of  the  current  system's  hardware  and  software.  It  will  replace  the 
current  obsolete  and  logistically  unsupportabie  equipment  with  commercial-off-the-shelf 
equipment  and  automate  many  of  the  current  manual  procedures.  The  upgrade  will  increase 
operational  effectiveness  in  several  critical  areas  by  taking  advantage  of  proven  and  emerging 
technology.  There  has  been  no  known  operational  test  and  evaluation  previously  performed  on 
the  SEON.  No  follow-on  test  is  planned. 

4.  The  following  Critical  Operational  Issues  (COIs)  were  developed  by  the  using  commands  and 
will  be  addressed  during  this  QOT&E: 

a.  COM:  Is  the  upgraded  SEON  safe  to  operate  and  maintain? 
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SECTION  I  -  INTRODUCTION 


l.d.  GENERAL.  Detachment  1.  Air  Force  Operational  Test  and  Evaluation  Center  (Del  I. 
AFOTEC)  will  conduct  a  Qualification  Operational  Test  and  Evaluation  iQOT&Hi  of  the 
Solar  Electro-Optical  Network  (SEON)  Upgrade/Replacement  Program  during  the  fourth 
quarter  of  fiscal  year  (FY)  93  and  the  first  quarter  of  FY  94  at  the  Palehua  Solar 
Observatory  in  Flawaii.  The  purpose  of  this  QOT&E  is  to  provide  information  to  decision 
makers  on  the  suitability  and  effectiveness  of  the  upgraded  SEON  for  milestone  111  decision 
maker  consideration  prior  to  upgrading  the  other  sites.  The  Test  Director  will  coordinate 
with  the  System  Program  Office  (SPO)  and  Headquarters  Air  Weather  Service  (HQ  AWS) 
on  all  proposed  changes  to  this  plan.  The  Commander.  Det  1.  AFOTEC  has  approval 
authority  for  all  changes  to  this  QOT&E  plan.  If  a  change  is  approved  prior  to  the  start  of 
the  test,  either  write-in  changes  or  page  changes  will  be  issued,  as  appropriate.  Once  the 
test  has  started,  all  changes  will  be  annotated  in  the  report. 

1.1.  SYSTEM  INFORMATION: 

1.1.1.  Background.  This  program  was  initiated  due  to  the  decreasing  maintenance 
supportability  and  limited  operational  effectiveness  of  the  current  system  hardware.  This 
program  will  replace  the  current  obsolete  and  logistical ly  unsupportable  equipment  with 
commercial-off-the-shelf  (COTS)  equipment,  and  automate  many  of  the  current  manual 
procedures  at  the  six  SEON  sites.  The  upgrade  will  increase  suitability  and  operational 
effectiveness  in  several  critical  areas  by  taking  advantage  of  proven  and  emerging 
technology.  There  has  been  no  known  operational  test  and  evaluation  previously  performed 
on  the  SEON.  Program  Management  Directive  X205(3)/PE35 1 1  IF.  19  March  1992,  directs 
Air  Force  Materiel  Command  (AFMC)  with  implementation  of  this  program.  The  Program 
Management  Plan,  30  March  19X9,  delegates  SPO  responsibility  to  Sacramento  Air 
Logistics  Center  (SM-ALC).  This  program  has  a  United  States  Air  Force  precedence  rating 
of  3-06  and  a  Force  Activity  Designator  (FAD)  of  111  for  acquisition.  SEON  has  an 
operational  precedence  rating  of  l-l  and  a  FAD  of  I.  Other  relevant  program  documents 
include  the  System  Operating  Requirements  Document  (SORD)  dated  10  August  19X9.  and 
the  Test  and  Evaluation  Master  Plan  (TEMP)  dated  I  April  1993. 

1.1.2.  Description: 

1.1. 2.1.  The  present  SEON  was  installed  in  the  mid-1970s,  utilizing  proven  technology  of 
solar  observing  systems  developed  in  the  1960s.  The  optical  anil  electronic  equipment 
were  designed,  manufactured,  and  individually  installed  by  a  small  group  of  scientists.  The 
SEON  is  the  only  current  or  planned  network  that  can  provide  optical  and  radio  solar 
observations  support  to  the  Department  of  Defense  (DOD).  It  measures  solar  events  to 
predict  their  impact  on  manned  space  activities  and  satellite  communications,  and  validates 
ionospheric  disturbances.  The  SEON  interfaces  with  the  Air  Force  Space  Forecast  Center 
(AFSFC)  and  the  Air  Force  Global  Weather  Central  (AFCiWC),  and  is  required  to  alert 
these  facilities  of  all  significant  solar  events  within  2  minutes  of  their  occurrence.  The 
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SEON  wasn't  designed  nor  intended  to  operate  in  a  high-threat  environment.  Air  Force 
weather  forecasters  and  observers  operate  the  SEON  under  the  control  of  the  AFSEC.  Air 
Force  weather  officers  stationed  at  Palehua  are  trained  in  computer  programming  and 
identify  software  problems,  recommend  software  changes,  and  perform  software 
maintenance  for  the  SEON.  The  SEON  hardware  is  maintained  by  Air  Force  weather 
equipment  technicians  (Air  Force  specialty  code  304X2)  from  the  local  supporting  major 
command:  e.g.,  United  States  Air  Forces  Europe  (USAFE).  Pacific  Air  Forces  (PACAF).  or 
Air  Combat  Command  (ACC).  Air  Force  Engineering  and  Technical  Services  (AFETS) 
personnel  from  Palehua.  Hawaii,  and  Holloman  Air  Force  Base  (AFB),  New  Mexico, 
provide  maintenance  training.  Each  site  has  developed  local  training  to  supplement  the 
AFETS  training.  Detachment  4,  AFSFC  will  provide  operations  training  to  all  sites. 


1.1. 2. 2.  The  SEON  is  comprised  of  five  Solar  Observing  Optical  Networks  (SOONs)  and 
four  Radio  Solar  Telescope  Networks  (RSTNs)  longitudinally  separated  to  ensure  the  sun  is 
constantly  monitored  (refer  to  Figure  1—1).  The  SOON  is  a  solar  optical  telescope 
(AN/FMQ-7)  with  automatic  solar  tracking.  The  RSTN  is  a  solar  radio  telescope 
(AN/FRR-95)  with  automatic  solar  tracking.  The  sites  at  Holloman  AFB,  New  Mexico, 
and  Ramey,  Puerto  Rico,  only  have  a  SOON.  The  site  at  Sagamore  Hill.  Massachusetts, 


only  has  a  RSTN.  The  sites  at  Palehua.  Hawaii;  Learmonth.  Australia;  and  San  Vito  Air 
Station,  Italy,  have  both  the  SOON  and  RSTN  and  are  referred  to  as  combined  sites.  A 
single  computer  is  currently  employed  at  each  solar  observatory  to  collect  and  analyze  the 
solar  data.  This  computer  also  measures  the  magnitude  of  solar  flares  and  prepares  the 
necessary  event  messages  for  release  by  the  operator.  The  proposed  upgrade  consists  of  the 
following  (refer  to  Figure  1—2  for  system  diagram): 

1.1. 2.2.1.  Replacement  of  the  two  video  cameras  in  the  SOON  with  charge-couplet!  device 
(CCD)  cameras.  The  CCDs  will  interface  with  the  new  SOON  computer  through  the 
existing  Digital  Imagery  Processing  Subsystem  (DIPS).  The  CCDs  will  also  interface  with 
the  VAX  4000  computer,  DIPS,  video  system,  television  monitors,  switches,  and 
distribution  amplifiers. 

1.1.2. 2.2.  Replacement  of  the  analog-to-digital  and  digitai-to-analog  converters,  relay 
drivers,  and  logic  cards  in  the  SOON  with  an  Intelligent  Data  Acquisition  System.  This 
will  allow  the  operation  of  the  SOON  to  continue  to  be  fully  automated. 

1.1.2.23.  Replacement  of  the  sweep  frequency  interferometric  radiometer  (SFIR)  in  the 
RSTN.  The  current  SFIR  collects  data  between  25—75  megahertz  (MHz)  while  the  new 
SFIR  will  collect  data  between  30—250  MHz.  The  new  SFIR  will  provide  better  real-time 
information  to  the  operator. 

1. 1. 2.2.4.  Replacement  of  the  computer  hardware.  Each  SOON  and  RSTN  will  have  its 
own  computer  system  as  shown  in  Figure  1-2.  A  thin- wire  Ethernet  will  interconnect  the 
individual  computers  and  the  terminal  server.  At  combined  sites,  the  computer  systems  will 
use  fiber  optics  to  interconnect  the  two  Ethernet  terminal  servers.  At  these  combined  sites, 
the  work  stations  must  be  able  to  provide  mutual  backup. 

1.1. 2.2.5.  An  uninterruptable  power  source  will  be  provided  to  ensure  stable  operation  of 
the  computer  systems.  It  must  provide  power  long  enough  for  backup  power  to  be 
established  or  the  system  safely  shutdown  without  the  loss  of  any  data. 

1.1. 2.2.6.  Upgrade  software  shall  improve  code  documentation  and  efficiency  while 
retaining  the  present  SEON  functions  as  described  in  the  SEON  and  RSTN  Functional 
Requirements  Descriptions  (FRDs). 

1.1.23.  SEON  is  a  part  of  the  Space  Environmental  Support  System  (SESS).  The  SESS 
observes  and  forecasts  environmental  conditions  and  system  effects  on  DOD  systems 
operating  more  than  50  kilometers  above  the  earth's  surface.  The  SESS  Forecast  Center  is 
currently  located  at  the  AFGWC  at  Offutt  AFB,  Nebraska  (refer  to  Figure  1—3). 

1.1. 2.4.  The  Palehua  Solar  Observatory  was  selected  for  the  QOT&E  because  it  is  a 
combined  SOON  and  RSTN  site  and  acts  as  the  depot  maintenance  and  software 
development  site  for  all  SEON  software  programs.  The  on-site  programmers  are  experts  on 
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the  system  and  fully  understand  its  operational  requirements.  Additionally,  the  required 
manpower  to  conduct  the  test  is  already  in  place,  saving  substantial  TDY  cost  over  the 
length  of  the  test. 


AN/FRR-95  RADIO  SOLAR  TELESCOPE  NETWORK  (RSTN) 


AN/FMQ-7  SOLAR  OBSERVING  OPTICAL  NETWORK  (SOON) 


FIGURE  1-2.  SEON  BLOCK  DIAGRAM 


a 


12.  OPERATIONAL  ENVIRONMENT.  The  SEON  is  operated  under  the  directum  of 
HQ  AWS.  Currently,  all  three  levels  of  maintenance  are  used  (organizational,  intermediate, 
and  depot);  however,  only  organizational-level  and  depot-level  maintenance  will  he  used  to 
support  all  new  equipment  resulting  from  this  program.  AFMC  will  provide  software 
configuration  management  and  control,  and  maintain  system  software.  Personnel  at 
Palehua,  Hawaii,  will  maintain  the  operational  software. 


FIGURE  1-3.  SE5S  INTERFACE 


1.2.1.  Threat  Summary.  The  SEON  is  not  expected  nor  designed  to  operate  in  a  hostile 
environment.  It  isn't  hardened  to  operate  in  a  chemical,  biological,  or  radiological 
environment.  Since  none  of  the  SEON  is  classified,  security  guides  are  not  required  for  the 
equipment,  details  of  equipment  performance,  or  design  specifications.  Communications 
security  has  been  considered  and  is  not  required. 

1.2.2.  Operational  Concept.  Each  site  is  required  to  operate  from  sunup  to  sundown. 

Their  location  allows  two  sites  to  concurrently  monitor  solar  activity,  increasing  the 
network's  overall  effectiveness.  The  effectiveness  of  the  SOON  is  degraded  by  any  object 
that  obscures  the  optical  path  between  the  telescope  and  the  sun;  e.g.,  clouds.  The 
effectiveness  of  the  RSTN  is  degraded  by  radio  frequency  interference  that  causes  false 
signals.  Each  SEON  site  reports  within  2  minutes  after  detection  of  all  significant  solar 


events  using  message  traffic  routed  over  the  Automated  Weather  Network  (AWN). 

AFSFC  and  AFGWC  use  the  information  to  forecast  the  effects  of  solar  activity  on  manned 
space  activities  and  satellite  communications. 

1.2.3.  Maintenance  Concept: 

1. 2.3.1.  Air  Force  personnel  in  the  304X2  career  field  will  maintain  the  SEON.  Air  Force 
Regulation  39—1 1  classifies  SEON  maintenance  as  a  special  duty.  The  Consolidated  Repair 
Activities,  which  currently  perform  the  intermediate-level  maintenance,  will  provide 
technical  guidance  and  maintenance  as  needed  on  the  new  equipment.  The  SOON 
Centralized  Repair  Activity  (CRA)  is  located  at  Holloman  AFB,  New  Mexico,  and  the 
RSTN  CRA  is  located  at  Palehua,  Hawaii.  Except  for  some  optics  conditioning  done  at 
Hill  AFB,  Utah,  all  depot  support  will  be  provided  by  the  CRAs. 

1. 2.3.2.  The  upgraded  SEON  will  not  require  peculiar  test  equipment  or  tools.  All  failures 
will  be  detected  using  test  points  and  existing  test  equipment.  Self-diagnostics  will  detect  a 
minimum  of  90  percent  ( % )  of  all  faults.  Replacing  a  single  line  replaceable  unit  (LRU) 
must  correct  a  minimum  of  90%  of  all  detected  faults,  replacing  no  more  than  two  LRUs 
must  correct  a  minimum  of  95%  of  all  detected  faults,  and  replacing  no  more  than  three 
LRUs  must  correct  a  minimum  of  99t7r  of  all  detected  faults. 

1. 2.3.3.  AFMC  will  do  software  configuration  management  and  control,  and  maintain 
system  software.  Personnel  at  Palehua,  Hawaii,  will  maintain  the  operational  software. 

1.2.3 .4.  The  supply  support  system  provides  for  sufficient  spares  to  allow  a  "remove  and 
replace"  maintenance  concept.  Spares  data  will  be  prepared  by  the  systems  contractor. 

This  data  will  identify  all  parts  used  in  the  fabrication  of  the  mechanical  and  electronic 
components.  From  this  data,  spares  to  support  the  QOT&E  will  be  procured.  For  the  first 
year  after  installation  of  the  system  upgrade,  a  warranty  will  be  in  effect  that  will  cover  all 
upgrade  components. 

1.2.4.  Training  Concept.  The  contractor;  Electrospace  Systems,  Incorporated  (ESI);  will 
provide  initial  operator  training  during  equipment  installation  and  checkout  at  each  site. 

The  new  operations  will  then  be  incorporated  into  the  existing  operators  training  currently 
taught  at  Holloman  AFB,  New  Mexico.  The  sites  will  conduct  refresher  training  using 
technical  orders  or  manuals.  The  Air  Force  Engineering  and  Technical  Services  personnel 
(AFETS)  will  continue  to  provide  ongoing  training  to  all  SEON  maintenance  personnel. 

ESI  will  provide  organizational-level  maintenance  training  to  technicians  and  AFETS. 

They  will  also  provide  depot-level  maintenance  training  to  the  CRAs  and  AFETS. 

1.3.  ACQUISITION  PROGRAM  STRUCTURE: 

1.3.1.  This  program  is  nondevelopmental.  The  program  will  incorporate  new  COTS 
hardware  with  nondevelopmental  software  (NDS)  and  operational  software.  The 
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operational  software  will  allow  the  operators  to  interface  with  the  different  NDS,  and 
control  the  SEON.  The  operational  software  will  be  written  in  the  Ada  programming 
language.  ESI  was  selected  by  competitive  bid  as  the  contractor  who  will  integrate  the  new 
hardware,  existing  hardware,  and  new  software  into  the  SEON. 

1.3.2.  The  contractor's  first  article  testing  must  be  completed  prior  to  the  start  of  QOT&E. 
QOT&E  is  scheduled  to  begin  during  the  second  quarter  of  FY  1993.  Table  1  —  1  is  the 
current  schedule  for  QOT&E. 


Table  1-1.  QOT&E  Program  Schedule  and  Milestones 


ACTIVITY 

DATES 

QOT&E  Plan 

June  93 

Installation  Completed 

June  93 

Conducting  Dedicated  QOT&E 

Aug  to  Nov  93 

QOT&E  Final  Report 

Jan  94 

Material  Improvement  Project  Review  Board 

Jan  94 

Milestone  III  Decision 


Jan  94 
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SECTION  2  -  OPERATIONAL  TEST  AND  EVALUATION  (OT&E)  OUTLINE 


2.0.  Critical  Operational  Issues  (COIs).  The  COIs  were  taken  directly  from  the  Solar 
Electro-Optical  Network  (SEON)  Test  and  Evaluation  Master  Plan  (TEMP),  I  Apr  93. 
These  nine  COIs  were  broken  into  specific  quantitative  or  qualitative  Measures  of 
Performances  (MOPs)  reflecting  key  parameters.  The  Test  Data  Scoring  Board  (TDSB) 
will  evaluate  results  before  assigning  ratings  to  the  MOPs.  The  TDSB  will  then  aggregate 
the  results  to  rate  the  COI  based  on  mission  impact.  All  MOPs  rated  "did  not  meet  user 
criteria"  will  be  evaluated  for  impact  upon  mission  accomplishment  before  assigning  an 
overall  rating  to  the  affected  COL  All  critical  MOPs  must  meet  user  requirements  in  order 
to  rate  the  associated  COI  as  "resolved  satisfactory."  In  addition,  MOPs  marked  as 
"significant"  were  determined  to  have  a  significant  impact  on  mission  accomplishment.  To 
cause  the  COI  to  be  rated  "resolved  unsatisfactory,"  the  results  of  these  MOPs  must  fail  to 
meet  user  criteria  and  prevent  mission  accomplishment.  The  Qualification  Operational 
Test  and  Evaluation  (QOT&E)  will  test  the  MOPs  against  documented  user  requirements 
and  will  result  in  one  of  the  following  two  ratings:  "Met  User  Requirements"  or  Did 
Not  Meet  User  Requirements."  The  final  report  will  use  the  MOP  findings  to  answer 
each  COI  as  "resolved  satisfactory"  or  "resolved  unsatisfactory."  Finally,  the  QOT&E  test 
report  will  use  the  answers  to  the  COIs  to  provide  the  Milestone  Decision  Authority  the 
following  statement:  "Based  upon  demonstrated  performance  during  QOT&E,  the  system 
is/is  not  operationally  effective/suitable." 

2.0.1.  The  COIs  are  as  follows: 


2.0. 1.1.  COI-1:  Is  the  upgraded  SEON  safe  to  operate  and  maintain? 

2.0. 1.2.  COI-2:  Do  the  SEON  computers  provide  sufficient  capabilities  to  support  the 
mission? 


2.0. 

1.3. 

COI-3: 

detection  of  solar 

2.0. 

1.4. 

COI-4: 

2.0. : 

1.5. 

COI-5: 

2.0. 

1.6. 

COI-6: 

2.0. 

1.7. 

COI-7: 

2.0.  i 

LX. 

COI-X: 

Does  the  upgraded  Radio  Solar  Telescope  Network  (RSTN)  allow 
events? 

Does  the  upgraded  SOON  allow  detection  of  solar  events? 

Is  the  SEON  software  supportable? 

Does  the  SEON  adequately  repoit  solar  events? 

Is  the  SEON  logistically  supportable? 

Does  the  SEON  facility  provide  adequate  power  and  environmental 


support? 


2.0. 1.9.  COI-9:  Is  the  human  factors  engineering  of  the  SEON  adequate? 
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2.0.2.  Objectives:  Objectives  are  listed  in  order  of  precedence.  The  "E"  or  ”S" 
preceding  the  number  indicates  the  objective  relates  to  either  the  system's  effectiveness  or 
suitability,  respectively. 

2.0.2. 1.  S-l.  The  safety  of  the  upgraded  Solar  Electro-Optical  Network  (SEON). 

2. 0.2.2.  S-2.  The  capabilities  of  the  SEON  computers  (includes  analysis  of  mutual 
backup,  restoral,  archival,  and  operational  capabilities). 

2. 0.2.3.  E-3.  The  capability  of  the  Radio  Solar  Telescope  Network's  (RSTNs)  swept 
frequency  interferometric  radiometer  (SFIR)  subsystem. 

2.0.2. 4.  E-4.  The  capability  of  the  RSTN's  discrete  frequency  radiometer. 

2.0.2. 5.  E-5.  The  capability  of  the  computer  to  interface  with  the  lock-in  amplifiers 

(LIAs). 

2.O.2.6.  E-6.  The  capability  of  the  Solar  Observing  Optical  Network's  (SOON)  charged 
coupled  devices  (cameras)  and  their  interfaces. 

2.O.2.7.  E-7.  The  capabilities  of  the  digital  imagery  processing  subsystem  (DIPS)  to 
perform  its  mission. 

2.O.2.X.  E-X.  The  capability  of  the  SOON's  analog  to  digital  (A/D)  converters. 

2.0.2. 9.  E-9.  The  capability  of  the  SOON's  digital  to  analog  (D/A)  converters. 

2.0.2.10.  E-10.  The  capability  of  the  interface  between  the  SOON's  computer  and  the 
DIPS. 

2.0.2.11.  S-II.  The  usability  of  the  SEON  software. 

2.0.2.12.  S-12.  The  maturity  of  the  SEON  software. 

2.0.2.13.  S-13.  The  logistics  supportability,  readiness,  and  maintainability  of  the  SEON. 

2.0.2.14.  E-14.  The  capability  of  the  network  to  transmit  and  receive  messages  over  the 
Automated  Weather  Network  (AWN). 

2.0.2.15.  S-15.  The  adequacy  of  the  technical  data  provided  with  the  upgraded  SEON. 

2.0.2.16.  E-16.  The  capability  of  the  system's  built-in  tests  (BIT)  to  support  mission 

needs. 
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2.0.2.17.  S- 17.  The  training  provided  to  support  the  upgraded  SEON. 

2.0.2.  IS.  E-1X.  The  capability  of  the  programmers  to  perform  their  mission. 

2.0.2.19.  E-19.  The  capability  of  the  uninterruptable  power  source  (UPS). 

2.0.2.20.  S-20.  The  maintainability  of  the  SF1R  software. 

2.0.2.21.  S-21.  The  maintainability  of  the  SEON. 

2.0.2.22.  S-22.  The  environmental  control  of  the  Palehua  Solar  Observatory. 

2.0.2.23.  S-23.  The  maintainability  of  the  discreet  frequency  radiometer  software. 
2.0.2.24.  S-24.  The  maintainability  of  the  SOON  software. 

2.0.2.25.  S-25.  The  support  resources  of  the  SEON  software. 

2.0.2.26.  S-26.  The  life-cycle  process  of  the  SEON  software. 

2.0.2.27.  S-27.  The  human  factors  engineering  (HFE)  aspects  of  the  upgraded  SEON. 

2.0.2.2X.  S-2X.  The  impact  of  electro-magnetic  interference  (EMI)/radio  frequency 
interference  (RF1)  on  SEON  operations. 

2.0.3.  COl/Objective  Matrix. 

CRITICAL  OPERATIONAL  ISSUES 
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COI/Objective  Matrix  Continued) 


CRITICAL  OPERATIONAL  ISSUES 
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2. 1.  SCOPE  AND  TEST  CONCEPT. 


2.1.1.  This  QOT&E  will  encompass  both  operational  effectiveness  and  suitat  'ity  issues 
by  performing  an  operational  evaluation  of  an  upgraded  SEON  site  comprised  of  both 
SOON  and  RSTN  subsystems.  The  test  will  be  representative  of  the  operational  scenarios 
users  are  expected  to  employ  during  actual  operations.  Solar  event  simulators  will  be  used 
during  Phase  1  to  provide  a  means  to  verify  proper  initial  operation  of  the  system.  These 
simulators  require  the  operators  and  the  system  to  perform  the  same  functions  they  will 
perform  during  normal  events.  During  Phase  2  and  3  of  the  test,  and  during  all  times 
when  specific  test  scenarios  are  not  being  performed,  the  test  'earn  will  observe  and 
evaluate  SEON  as  it  performs  its  primary  mission  of  solar  patrol.  There  are  no  differences 
in  the  mission  SEON  will  perform  during  test,  and  that  which  it  is  expected  to  perform 
during  normal  operations.  All  key  user  operational  issues  will  be  addressed  during  this 
QOT&E.  The  test  will  be  conducted  over  a  90-day  period,  and  will  test  the  upgraded 
equipment  and  software  and  their  effect  on  the  mission  capability  of  the  SEON.  Testing 
will  be  conducted  during  normal  SEON  operational  duty  hours  which  are  sunup  to 
sundown  (15  hours)  each  day  for  a  total  1350  hours.  The  90  days  will  be  broken  down 
into  three  30-day  phases:  Simulations,  Normal  Operations,  and  Software.  Phase  I, 
Simulations,  conducted  during  the  first  30  days,  will  consist  of  initial  system  start-up.  solar 
event  simulations,  software  and  hardware  maintainability,  and  a  safety  evaluation.  Phase 
2.  Normal  Operations,  conducted  during  the  middle  30  days,  will  evaluate  their  ability  to 
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perform  the  mission  using  real-time  solar  data.  This  operational  solar  data  will  lx- 
compared  to  the  solar  data  simultaneously  obtained  by  other  SEON  locations  to  determine 
the  operational  effectiveness  of  the  upgraded  system.  This  phase  also  includes  normal 
maintenance  and  the  evaluation  of  the  uninterruptable  power  source.  Phase  3.  Software, 
will  evaluate  the  on-site  programmers  ability  to  perform  depot  maintenance  on  the  system 
application  software.  This  phase  also  includes  human  factors  engineering,  and  software 
usability.  Reliability.  Maintainability,  and  Availability  (RMA)  data  will  be  collected 
during  the  entire  test  period  and  will  be  used  to  determine  the  operational  suitability  of 
the  SEON.  No  specific  test  limitation  has  been  identified.  Data  reduction  and  data 
analysis  will  occur  daily  during  the  test.  The  daily  test  schedule  has  2  hours  per  day 
dedicated  to  these  activities.  The  user.  Air  Weather  Service,  expects  a  representative 
sample  of  solar  events  to  occur  during  the  test.  The  following  schedule  outlines  our  plan: 

PHASE  ONE  -  SIMULATIONS: 


NOTE:  Test  start  day  =  fS+O 


Test  Objective 

S-tart 

Number  Davs  Scheduled 

Initial  Support  Supply  Listing  (ISSL) 

TS-fO 

1  day 

Safety  Evaluation 

TS+O 

1  day 

System  Start-up 

TS+O 

4  days 

Software  Maintainability 

TS+O 

10  days 

Computer  Operations 

TS+5 

IS  days 

RSTN  Simulations 

TS+7 

6  days 

SOON  Flare  Simulations 

TS+7 

7  days 

Resolution  Target 

TS+17 

2  days 

DIPS  Interface 

TS+25 

3  days 

Faults 

TS+2X 

3  days 

PHASE  TWO  -  NORMAL  OPERATIONS: 

Test  Objective 

SMI 

Number  Davs  Scheduled 

Normal  Patrol  Duties 

TS+31 

30  days 

Preventative  Maintenance  Inspections 

TS+33 

7  days 

Uninterruptable  Power  Source  (UPS) 

TS+40 

7  days 

PHASE  THREE  -  SOFTWARE: 

Test  Objective 

Start 

Number  Davs  Scheduled 

Usability 

TS+61 

7  days 

HFE  Survey 

TS  +62 

1  day 

PHASE  THREE  -  SOFTWARE  (CONTINUED): 


Test  Objective 

Start 

Number  Davs  Scheduled 

Support  Resources 

TS+6S) 

4  days 

Programming  Scenarios 

TS+75 

15  days 

NOTE:  Testing  will  usually  occur  during  the  normal  operating  hours  of  the  site  which  is 
open  from  sunup  to  sundown.  This  equates  to  a  15-hour  duty  day  for  the  site  during  the 
test  period. 

2.2.  PLANNING  CONSIDERATIONS  AND  LIMITATIONS: 

2.2.1.  Planning  Considerations: 

2.2. 1.1.  The  limited  length  of  the  QOT&E  may  restrict  high-confidence  level,  statistical 
verification  of  logistics  supportability,  readiness,  and  maintainability  results.  Because  of 
this,  the  user  agreed  to  rate  the  results  of  the  reliability  and  maintainability  evaluations 
from  the  performance  data  gathered,  disregarding  a  lack  of  statistical  confidence. 

2.2. 1.2.  If  actual  solar  events  do  not  occur,  the  charge-coupled  devices  (CCDs)  and  solar 
telescope  will  be  tested  using  stored  and  artificial  event  images  to  simulate  solar  activity. 
These  images  will  be  introduced  into  the  SOON  and  will  allow  conclusions  to  be  made 
about  their  capabilities  from  data  collected. 

2.2. 1.3.  Because  of  the  limited  length  of  the  QOT&E.  all  possible  solar  events  are  not 
expected  to  occur.  This  may  prevent  a  high  confidence  that  the  upgraded  system  can 
detect  all  types  of  solar  activity.  Because  of  this,  the  users  have  agreed  that  the  system 
effectiveness  should  be  rated  based  on  the  data  gathered  operationally,  and  on  data 
gathered  through  the  various  simulations. 

2.2. 1.4.  If  operational  commitments  conflict  with  the  scheduled  test  events,  delays  in  the 
QOT&E  may  occur. 

2.2. 1.5.  In  the  case  of  a  SEON  system  failure,  the  Test  Director  will  determine  if  the 
contractor  will  be  allowed  to  make  any  modifications  or  adjustments.  If  a  modification  or 
adjustment  is  made,  its  effect  on  the  previously  collected  data  must  be  carefully 
considered.  Modifications  that  invalidate  enough  data  to  prevent  conclusive  results  will 
not  be  allowed. 

2.2.2.  Limitations.  The  sun  is  currently  in  a  time  of  decreased  activity  and  all  solar 
events  may  not  occur  during  the  time  of  the  QOT&E.  The  user.  AWS,  expects  a 
representative  sample  of  solar  events  to  occur  during  the  test  period  and  cannot  identify, 
before  hand,  a  specific  event  that  will  not  occur.  Therefore,  there  are  no  known  test 
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identifiable  before  test  start.  All  COIs  will  be  resolved  as  a  result  of  this  QOT&E. 

2.3.  SYSTEM  CONTRACTOR  INVOLVEMENT  IN  DEDICATED  OT&E.  There  will 
be  no  system  contractor  involvement  during  this  QOT&E  except  as  indicated  in  paragraph 
2.2. 1.5.  above. 

2.4.  OT&E  SCHEDULE  AND  READINESS  REQUIREMENTS: 

2.4.1.  The  following  is  the  schedule  for  QOT&E  events: 

2.4. 1.1.  Planning  the  QOT&E  will  be  ongoing.  The  following  activities  are  scheduled  or 
have  taken  place: 

2.4. 1.1.1.  Nov  91:  the  test  support  group  (TSG)  met  at  the  Palehua  Solar  Observatory  in 
Hawaii.  During  the  meeting,  COIs,  objectives,  and  user  criteria  were  developed  for 
submission  to  the  System  Program  Office  (SPO)  for  development  of  the  TEMP. 

2.4. 1.1.2.  Mar  92:  the  TSG  met  at  the  Air  Force  Space  Forecast  Center  (AFSFC)  at 
Falcon  Air  Force  Base  (AFB),  Colorado.  The  draft  QOT&E  plan  was  reviewed  by  the 
users  and  SPO  focusing  on  testing  methodology  and  acceptable  criteria.  Action  items  were 
assigned  to  the  appropriate  agency  for  resolution. 

2.4. 1.1.3.  Mar  92:  the  TSG  met  at  McClellan  AFB,  California,  to  determine  procedures 
for  aggregation  of  the  test  results  for  rating  the  applicable  critical  operational  issues.  The 
draft  QOT&E  plan  was  reviewed  by  the  using  command  and  SPO.  Action  items  were 
assigned  to  the  appropriate  agency  for  resolution. 

2.4. 1.1.4.  Aug  92:  TSG  meeting  was  held  at  Sacramento  Air  Logistics  Center  to  develop 
a  method  for  aggregating  the  MOP  results  to  rate  the  COIs. 

2.4. 1.1.5.  Nov  92:  the  contractor  first  article  test  (CFAT)  was  performed  at  Electrospace 
Systems,  Incorporated  (ESI)  in  Richardson,  Texas.  This  test  will  be  considered  the 
qualification  test  and  evaluation,  and  will  validate  that  the  technical  specifications  were 
met. 

2.4. 1.1.6.  Nov  92:  QOT&E  test  plan  was  submitted  for  approval. 

2.4. 1.1.7.  Dec  92:  the  TSG  will  meet  at  the  Palehua  Solar  Observatory  in  Hawaii  to 
finalize  the  detailed  test  procedures. 

2.4.L1.X.  May  93:  the  installation  of  the  upgrade  at  the  Palehua  Solar  Observatory  in 
Hawaii  completed. 

2  4.1.2.  Jan  -  Nov  93:  QOT&E  will  occur.  The  following  activities  are  scheduled  or 


have  taken  place: 


2.4. 1.2.1.  Feb  to  Jun  93:  the  software  life-cycle  evaluation  data  collection  at  Sacramento 
Air  Logistics  Center  (SM-ALC)  at  McClellan  AFB,  California;  the  Palehua  Solar 
Observatory,  Hawaii;  and  ESf  in  Richardson.  Texas.  Results  of  the  evaluation  will  be 
published  in  the  final  QOT&E  report. 

2.4. 1.2.2.  Jul  93:  contractor  maintainability  demonstration  at  the  Palehua  Solar 
Observatory,  Hawaii. 

2.4. 1.2.3.  Jul  93:  certification  that  the  Palehua  Solar  Observatory  SEON  is  ready  for 
dedicated  QOT&E  by  the  SPO.  This  formally  releases  control  of  the  system  to  the 
operational  test  agency  (OTA). 

2.4. 1.2.4.  Jul  93:  software  maintainability  evaluation  data  collection  at  the  Palehua  Solar 
Observatory,  Hawaii.  Results  of  the  evaluations  will  be  published  in  the  final  QOT&E 
report. 

2.4. 1.2.5.  Aug-Nov  93:  dedicated  QOT&E  at  the  Palehua  Solar  Observatory,  Hawaii. 

Data  required  to  support  the  COIs  for  the  SEON  will  be  gathered.  Product  quality 
deficiency  reports  will  be  submitted  in  accordance  with  (1AW)  Technical  Order  (T.O.) 
00-35D-54  on  all  verified  discrepancies.  The  following  QOT&E  events  will  also  take 
place  during  this  time: 

2.4. 1.2.5. 1.  Aug-Nov  93:  software  usability  and  support  resources  evaluations  data 
collection.  Results  of  the  evaluations  will  be  published  in  the  final  QOT&E  report. 

2.4. 1.2.5. 2.  Nov  93:  test  data  scoring  board  (TDSB)  will  convene  at  the  test  site  to 
validate  the  reliability  and  maintainability  data. 

2.4. 1.2.5. 3.  Aug-Dec  93:  data  reduction  at  Palehua  Solar  Observatory  on  a  daily  basis  and 
at  Scott  AFB,  Illinois,  during  phase  2  of  the  test  and  after  the  last  test  event  at  Palehua. 

All  data  collected  will  be  reduced  and  validated.  Results  will  be  published  in  the  QOT&E 
report. 

2.4. 1.3.  Dec  93- Jan  94:  QOT&E  reporting  will  occur.  During  this  period,  the  following 
activities  are  scheduled: 

2.4. 1.3.1.  Jan  94;  material  improvement  project  review'  board  hosted  by  SM-ALC  at 
McClellan  AFB,  California.  This  board  will  take  action  IAW  T.O.  00-35D-54. 

2.4. 1.3.2.  Jan  94:  QOT&E  report  board  at  Scott  AFB,  Illinois.  The  report  board  will 
review  the  document  for  proper  format  and  philosophy.  All  conclusions  and 
recommendations  will  be  reviewed  to  ensure  they  are  supported  by  the  findings.  After  the 
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recommendations  will  be  reviewed  to  ensure  they  are  supported  by  the  findings.  After  the 
required  coirections  are  made,  the  QOT&E  report  will  be  forwarded  for  final  approval. 

2.4. 1.3.3.  Jan  94:  QOT&E  report  printing  and  distribution  at  Scott  AFB,  Illinois.  The 
final  report  will  be  printed  and  distribution  made. 

2.4.2.  The  following  actions  must  be  completed  before  the  QOT&E  can  take  place: 

2.4.2. 1.  The  SPO  must  develop  and  publish  an  approved  TEMP. 

2. 4.2.2.  The  OTA  must  approve  the  QOT&E  plan. 

2. 4.2.3.  The  SPO  must  ensure  the  SEON  at  the  Palehua  Solar  Observatory  is  upgraded. 

2. 4.2. 4.  The  SPO  must  ensure  the  CFAT  is  completed  with  satisfactory  results. 

2. 4.2.5.  The  Commander  of  the  Palehua  Solar  Observatory  must  have  the  upgraded  SEON 
system  safety  inspected. 

2.4.2. 6.  The  SPO  must  provide  the  OTA  with  a  written  certification  that  the  upgraded 
SEON  at  the  Palehua  Solar  Observatory  is  ready  for  dedicated  QOT&E. 

2.4.3.  RESOURCE  SUMMARY.  The  upgraded  SEON  at  the  Palehua  Solar  Observatory, 
with  the  following  personnel  and  material,  are  required  to  support  this  QOT&E: 

Personnel 

I  Test  Director  from  the  OTA 
1  Associate  Test  Director  from  the  OTA 
1  Software  Test  Manager  (STM)  from  the  OTA 
5  Associate  STMs  from  the  OTA  (as  required) 

1  Data  Analyst  from  the  OTA  (as  required) 

2  SEON  programmers  from  Air  Weather  Service  (AWS) 

4  Weather  forecasters  from  AWS 

I  Information  manager  from  AWS  (as  required) 

1  Air  Force  Engineering  and  Technical  Services  (AFETS)  from  Air 
Combat  Command  (ACC) 

4  Equipment  technicians  from  the  Pacific  Air  Forces  (PACAF) 

1  AFETS  from  PACAF 

1  Systems  expert  from  IPS  radio  space  services 
1  Software  consultant  from  Phillips  Labs 


Required  material  not  part  of  the  SEON-1  upgrade. 

2  SOON  Flare  Simulators 

3  Hydrothermographs 

1  Current  SF1R  with  chart  recorder 
1  Notebook  personal  computer  (PC) 

1  Facsimile  Machine 
1  Desktop  PC 

Test  Articles 

SOON; 

4  VAX  3100-76  Workstations 

1  VAX  4000/300  Real  Time  Processor 
1  LN05R  Laser  Printer 
1  H-Alpha  CCD  Camera 
1  Spectrographic  CCD  Camera 
1  Digital  Imagery  Processing  Subsystem  (DIPS) 
1  AN/FMQ-7  Solar  Optical  Telescope 
1  Camera  Controller 
1  Intelligent  data  acquisition 
1  Uninterruptable  Power  Source 

1  Ethernet  terminal  server 

RSTN: 

4  VAX  3100-76  Workstations 

2  VAX  3305  Real  Time  Processors 
1  LN05R  Laser  Printer 

16  Model  5207  Lock-in  amplifiers 
1  Swept  Frequency  Interferometric  Radiometer 
1  AN/FRR-95  Solar  Radio  Telescope 
X  Discrete  Frequency  Radiometers 
I  Plotter 

1  Uninterruptable  Power  Source 
1  Ethernet  terminal  server 

Communications: 

1  AWN  interface 
1  SELSIS  interface 
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SECTION  3  -  METHODOLOGY 


3.0.  GENERAL.  This  section  describes  the  Qualification  Operational  Test  and  Evaluation 
(QOT&E)  operational  effectiveness  and  suitability  test  activities  that  will  be  accomplished 
by  the  test  team  during  the  Solar  Electro-Optical  Network  (SEON)  QOT&E.  The  Measures 
of  Performance  (MOPs),  which  are  a  quantitative  or  qualitative  measure  of  a  system's 
capabilities  or  characteristics,  were  derived  from  and  support  the  Critical  Operational  Issues 
(COls).  The  COIs  are  questions  about  critical  operational  issues  concerning  a  system’s 
essential  capabilities,  risks,  or  uncertainties  which  must  be  answered  before  the  overall 
worth  of  the  new  or  modified  system  can  be  estimated.  The  user.  Air  Weather  Service,  has 
identified  18  key  parameters  that  will  be  fully  tested  during  this  QOT&E. 

3.0.1.  COl  Summary: 

3.0. 1.1.  The  following  matrix  correlates  the  various  test  and  evaluation  events  to  the  COIs 
they  support. 


Test 

Phase 

Test  Event 

COI 

Supported 

Simulations 

Safety 

1 

(day  1  -  30) 

Initial  Support  Supply  Listing 

7 

System  Start-up 

2 

Software  Maintainability 

3 

Computer  Operations 

2 

RSTN  Simulations 

3,6 

SOON  Flare  Simulations 

4,6 

Resolution  Target 

4 

DIPS  Interface 

2,3 ,4 ,5 

Faults 

2,7 

i-i 


COl-1 


MOP  1-1:  number  of  SEON  safety  hazards  that  meet  category  (CAT)  I 
deficiency  PQDR  criteria. 


MOP  1-2:  number  of  SEON  safety  hazards  that  meet  CAT  II  deficiency  PQDR 
criteria. 


COI-2  MOP  2-1:  number  of  real-time  computer  processors  that  cannot  be  restored  using 

the  Ethernet. 

MOP  2-2:  number  of  workstations  that  cannot  be  restored  to  full  mission 
capability  from  other  nodes  over  the  ethernet. 

MOP  2-3:  number  of  computers  that  cannot  be  restored  to  full  mission  capability 
using  the  tape  back-ups. 

MOP  2-4:  number  of  interfaces  not  achieved  between  the  computers  and 
equipment. 

MOP  2-5:  number  of  menu  items  that  cannot  be  selected  by  a  single  keystroke  or 
a  single  combination  of  two  keystrokes  (as  in  alt/F10). 

MOP  2-6:  number  of  mission  data  segment  changes  not  executable  via  operator 
menu  selection. 

MOP  2-7:  number  of  equipment  checks  and  alignments  not  executable  via 
keyboard  entry. 

MOP  2-8:  percent  of  selected  text  files  adequately  printed  to  the  LN05R  printers. 

MOP  2-9:  percent  of  selected  graphics  files  adequately  printed  to  the  LNO.SR 
printers. 

MOP  2-1(1:  percent  of  screen  images  adequately  printed  to  the  LN05R  printers. 

MOP  2-11:  number  of  inadequate  plots  from  the  Sweep  Frequency  Interferometric 
Radiometer  (SFIR). 

MOP  2-12:  number  of  Digital  Command  Language  (DCL)  level  functions  that 
cannot  be  performed  by  the  operators. 

MOP  2-13:  number  of  nodes  that  cannot  be  assigned  or  reassigned  without 
bringing  the  system  down. 

MOP  2-14:  uptime  ratio  (UTR)  of  the  RSTN  computer  system. 


MOP  2-15:  UTR  of  the  SOON  computer  system. 


COI-2  MOP  2-16:  time  required  tor  operators  to  ioad  and  run  new  version  software. 

(Continued) 

MOP  2-17:  performs  required  functions  where  the  computer  individually  reads, 
writes,  interacts  and  controls  for  each  L1A. 

MOP  2-1X:  percent  of  faults  detected  hy  the  automatic  monitoring  system. 
MOP  2-19:  false  alarm  rate  of  faults  hy  the  automatic  monitoring  system. 

MOP  2-20:  number  of  incorrect  status  indications. 

MOP  2-21:  number  of  work  stations  assigned  as  system  console,  that  cannot 
perform  calibration  of  the  DIPS. 

MOP  2-22:  number  of  work  stations  assigned  as  system  console,  that  cannot 
control  the  data  image  collection  function  of  the  DIPS. 

MOP  2-23:  number  of  work  stations  assigned  as  system  console,  that  cannot 
control  the  scheduling  of  movie  replay  function  of  the  DIPS. 

MOP  2-24:  number  of  workstations  assigned  as  system  console,  that  cannot 
control  the  remote  shutdown  of  the  DIPS. 


MOP  2-25:  number  of  inadequate  magnetic  contour  and  Stoneyhurst  plots 
performed  by  the  system  using  data  provisioned  from  the  telescopes  under 
control  of  the  workstations. 


COI-3  MOP  3-1:  percent  of  solar  hursts  detected  by  the  current  SFIR  between 

30-75  megahert/  (MH/)  that  are  also  detected  by  the  new  SFIR  between 
30-75  MH/  while  both  SFIRs  are  operational. 

MOP  3-2:  percent  ol  solar  bursts  detected  by  other  SFIRs  (at  other  SEON  sites) 
between  30-75  MHz  that  tire  also  detected  by  the  new  SFIR. 

MOP  3-3:  subjective  rating  by  qualified  analysts  ol  solar  sweep  signatures 
detected  between  30-75  MHz. 

MOP  3-4:  subjective  rating  by  qualified  analysts  of  solar  sweep  intensities 
between  30-75  MHz. 

MOP  3-5:  speed  of  Type  2  shocks. 

MOP  3-6:  minimum  sensitivity  of  the  new  SFIR  at  30  MHz. 

MOP  3-7:  minimum  sensitivity  of  the  new  SFIR  at  250  MHz. 

MOP  3-8:  number  of  solar  events,  types  of  solar  burst  signatures,  and  bursts 
intensities  between  75-250  MHz  detected  by  the  SFIR. 

MOP  3-9:  percent  of  solar  intensity  difference  between  that  indicated  by  the 
discrete  frequency  radiometer  computer  and  the  charted  intensity. 

MOP  3-10:  the  computer  reported  burst  characteristics  (start  time,  maximum 
time,  stop  time,  mean  flux,  integrated  flux,  and  peak  flux)  compared  to  the  charted 
characteristics. 

MOP  3-1 1:  number  of  solar  hursts  the  discrete  frequency  radiometer  incorrect 
identifies  (excluding  Radio  Frequency  Interference  (RFI)). 


MOP  3-12:  subjective  rating  by  experienced  RSTN  analysts  and  programmers  of 
discrete  frequency  bursts. 


MOP  4-1;  subjective  rating  by  qualified  analysts  of  H-Alpha  images. 

MOP  4-2;  minimum  resolution  of  large  scale  H-Alpha  images. 

MOP  4-3:  minimum  resolution  of  full  disk  H-Alplta  images. 

MOP  4-4:  subjective  rating  by  qualified  analysts  of  spectrograplnc  images. 

MOP  4-5:  minimum  resolution  of  large  scale  spectrograplnc  images. 

MOP  4-ft:  minimum  resolution  of  lull  disk  spectrograplnc  images. 

MOP  4-7:  subjective  rating  by  qualified  analysts  of  H-Alpha  images  transferred  to 
the  DIPS  from  the  RSI 70  output  of  the  charge-coupled  device  (CCD). 

MOP  4-X:  subjective  rating  by  qualified  analysts  of  spectrograplnc  magnetic  data 
transferred  to  the  DIPS. 

MOP  4-9:  number  of  frames  transferred  to  the  DIPS  from  the  SOON. 

MOP  4-10:  DIPS  playback  speed. 

MOP  4-11:  number  of  frames  that  can  he  stored  by  the  DIPS. 

MOP  4-12:  resolution  of  each  frame  (stored  in  the  DIPS). 

MOP  4-13:  sufficiency  of  shades  of  gray  (of  stored  DIPS  images). 

MOP  4-14:  speed  of  overlaying  pictures  on  stored  grids  (in  the  DIPS). 

MOP  4-15:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
histogram  capability. 

MOP  4-16:  minimum  resolution  of  H-Alpha  and  visible  light  including 
magnesium  (Mg  B,). 

MOP  4-17:  resolution  of  the  detected  solar  magnetic  field. 

MOP  4- IX:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
image  enhancement  of  gray  scale  and  pseudo  color. 

MOP  4-19:  rate  of  optical  scan  conversion. 

MOP  4-2(1:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
adequacy 
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MOP  4-21:  subjective  ratine  by  trained  National  Geophysical  Data  Center 
(NGDC)  analysts  of  the  adequacy  ol  archived  solar  data. 

MOP  4-22:  subjective  rating  by  trained  SOON  analysts  and  programmers  ol  the 
speetrographic  output  data. 

MOP  4-23:  subjective  rating  hy  trained  SOON  analysts  and  programmers  o'  the 
digital  images  (scaled,  standardized,  and  superimposed). 

MOP  4-24:  amount  of  degradation  in  current  provided  operability. 

MOP  4-25:  perforins  required  Analog  to  Digital  (A/D)  converter  functions. 

MOP  4-26:  number  of  SOON  significant  outages  caused  hy  an  A/D  converter. 

MOP  4-27:  number  of  A/D  converters  failures. 

MOP  4-28:  performs  required  Digital  to  Analog  (D/A)  converter  functions. 

MOP  4-29:  number  of  SOON  significant  outages  caused  by  a  D/A  converter. 
MOP  4-30:  number  of  D/A  converter  failures. 


MOP  4-31:  subjective  rating  by  experienced  SOON  analysts  and  programmers  of 
solar  flare  analysis  function. 
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MOP  5-1:  usability  of  the  SEON  software. 
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MOP  5-2:  maturity  of  the  SEON  software. 

MOP  5-4:  niainiaiitahility  of  the  SEON  software. 

MOP  5-4:  support  resources  of  the  SEON  software. 

MOP  5-5:  life-cycle  process  of  the  SEON  software. 

MOP  5-6:  time  required  to  update  and  install  completely  new  source  software 
distribution  into  the  system. 

MOP  5-7:  number  of  approved  software  changes  the  programmers  cannot 
introduce  into  the  application  software. 

MOP  5-8:  system  impact  of  testing  software  changes. 

MOP  5-9:  number  of  tested  software  changes  that  cannot  he  introduced  into  the 
system  without  rebooting  the  computer. 

MOP  5-10:  number  of  software  development  tools  the  programmers  cannot 
effectively  use. 

MOP  5-11:  number  of  program  documentation  items  the  programmers  cannot 
update. 

MOP  5-12:  number  of  programming  tasks  that  cannot  be  performed  due  to 
inadequate  training. 

MOP  6-1:  percent  of  message  formats  transmitted  1AW  Air  Force  Space  Forecast 
Center  Pamphlet  (AFSFCP)  105-5. 

MOP  6-2:  time  between  solar  activity  and  transmission  of  the  required  message. 

MOP  6-5:  number  of  errors  between  transmitted  and  received  messages. 

MOP  6-4:  number  of  solar  events  the  analysts  are  unable  to  respond  to  within 
time  constraints  listed  in  Air  Force  Space  Command  Regulation. 
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MOP  7-1: 


mean  lime  between  critical  l.ulnre  (MTB('F)  ul  the  RSFN. 


MOP  7-2:  MTBCF  ol  the  SOON. 

MOP  7-3:  mean  time  between  maintenance  corrective  (MTBMcj. 

MOP  7-4:  mean  repair  tune  (MKT)  of  the  SEON. 

MOP  7-5:  MRT  ot  the  embedded  computer  resources. 

MOP  7-6:  MRT  of  the  SFIR. 

MOP  7-7:  Max  time  to  repair  (Max  TTR)  lor  the  SFIR. 

MOP  7-8:  Max  TTR  for  the  CCD. 

MOP  7-9;  Max  TTR  for  the  UPS. 

MOP  7-10:  Max  TTR  for  the  SEON. 

MOP  7-11:  number  of  failures  not  detected  through  test  points  using  full 
resources. 


MOP  7-12:  percent  of  detected  faults  corrected  by  replacing  one  line  replaceable 
unit  (LRU). 


MOP  7-13:  percent  of  detected  faults  corrected  bv  replacing  a  maximum  of  two 
LRUs. 

MOP  7-14:  percent  of  detected  faults  corrected  by  replacing  a  maximum  of  three 
LRUs. 

MOP  7-15:  numher  of  special  tools  required  to  maintain  the  SEON. 

MOP  7-16:  number  of  peculiar  test  equipment  required  to  maintain  the  SEON. 

MOP  7-17:  weekly  preventive  maintenance  inspection  (PMil  times  for 
maintaining  the  upgraded  portions  of  the  SEON. 

MOP  7-1  8:  weekly  PMI  times  for  maintaining  the  upgraded  portions  of  the 
RSTN, 

MOP  7-19:  amount  of  increased  time  required  to  perform  weekly  PMI  for 
maintaining  the  non-upgraded  portions  of  the  RSTN. 


MOP  7-2(1  amount  of  increased  time  required  to  perform  weekly  PMI  for 
maintaining  the  non-upgraded  portions  of  the  SOON. 
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MOP  7-21:  number  and  skill  level  of  personnel  required  lo  perform  any  SEON 
maintenance  routine. 

MOP  7-22:  number  ol  failures  requiring  more  than  removal  and  replacement  of 
defective  LRUs  on  the  new  SFIR.  CCDs.  IDAS,  or  computer  resources. 

MOP  7-23:  depot-level  task  completion  time. 

MOP  7-24:  MTBCF  of  the  SFIR  and  discrete  frequency  radiometer. 

MOP  7-25:  percent  of  initial  spares  support  listing  (ISSL)  items  filled  at  site  prior 
to  QOT&E. 

MOP  7-26:  percent  of  critical  ISSL  items  filled  at  site  prior  to  QOT&E. 

MOP  7-27:  number  of  significant  outages  caused  by  erroneous  technical  data. 

MOP  7-2K:  subjective  assessment  by  qualified  programmers,  technicians,  and 
analysts  of  the  technical  data. 

MOP  7-29:  number  of  significant  outages  caused  by  inadequate  training. 

MOP  7-30:  subjective  assessment  by  qualified  SEON  operators,  maintainors,  and 
analysts  of  the  training. 

MOP  7-31:  number  of  discrepancies  between  maintenance  AFSC  job  descriptions 
and  tiie  tasks  performed. 

MOP  7-32:  adequacy  of  the  course  chart  for  maintenance. 

MOP  7-33:  adequacy  of  plan  of  instruction  for  maintenance. 

MOP  7-34:  adequacy  of  the  provided  student  maintenance  training  material. 

MOP  7-35:  adequacy  of  the  maintenance  training  program  and  training  equipment 
plan. 

MOP  7-36:  number  of  discrepancies  between  operations  AFSC  job  descriptions 
and  the  tasks  performed. 

MOP  7-37:  adequacy  of  the  course  chart  for  operations. 
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MOP  7-3X:  adequacy  of  plan  of  instruction  lor  operations. 


MOP  7-31):  adequacy  of  the  provided  student  operations  training  material. 

MOP  7-40:  adequacy  of  the  operations  training  program  and  training  equipment 
plan. 

MOP  7-41:  UTR  of  the  SOON  system. 

MOP  7-42:  UTR  of  the  RSTN  system. 


3.1.  COI-1.  Is  the  upgraded  SEON  safe  to  operate  and  maintain? 

3.1.1.  Scope.  The  safety  of  operations  and  maintenance  of  the  SEON  will  be  evaluated 
during  the  entire  90-day  Qualification  Operational  Test  and  Evaluation  (QOT&E).  MOP 
1-1  is  the  critical  MOP  supporting  this  COI.  The  site  safety  officer  will  perform  a  safety 
evaluation  prior  to  operation  of  the  SEON  equipment, 

3.1.2.  Measures  of  performance  (MOPs)  and  Performance  Criterion. 


3. 1.2.1.  MOP  1-1:  number  of  SEON  safety  hazards  that  meet  category  (CAT)  1  deficiency 
PQDR  criteria.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

3. 1. 2.2.  MOP  1-2:  number  of  SEON  safety  hazards  that  meet  CAT  II  deficiency  PQDR 
criteria.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.1.3.  Mission  Scenarios.  During  normal  operations  and  maintenance,  any  safety 
deficiency  will  be  immediately  reported  to  the  Site  Safety  Officer  for  full  investigation. 

The  Site  Safety  Officer  will  fully  document  the  deficiency  and  provide  this  documentation 
to  the  Test  Director.  The  Test  Director  will  ensure  the  required  PQDR  is  released  in 
accordance  with  (IAW)  Technical  Order  (T.O.)  00-35D-54. 

3.1.4.  Method  of  Evaluation.  The  number  of  CAT  1  and  CAT  II  safety  PQDRs  generated 
will  be  compared  to  the  performance  criterion.  MOP  1-1  was  determined  to  be  critical  to 
mission  accomplishment  and  must  meet  user  criteria  to  rate  the  COI  as  "resolved 
satisfactory".  MOP  1-2  was  marked  as  "significant"  and  was  determined  to  have  a 
significant  impact  on  mission  accomplishment.  To  cause  the  COI  to  be  rated  "resolved 
unsatisfactory",  the  results  of  MOP  1-2  must  fail  to  meet  user  criteria  and  prevent  mission 
accomplishment. 

3.2.  COI-2:  Do  the  SEON  computers  provide  sufficient  capabilities  to  support  the 
mission? 

3.2.1.  Scope.  The  capabilities  of  the  SEON  compu^rs,  the  computer's  built-in  tests  (BIT), 
the  interface  to  the  RSTN’s  lock  in  amplifiers  (LIAs),  and  the  interface  to  the  digital 
imagery  processing  subsystem  (DIPS)  will  be  evaluated  during  normal  operation  and 
maintenance  of  the  SEON  system.  MOPs  2-1,  2-2,  2-3,  2-4,  2-13,  and  2-16,  are  the  critical 
MOPs  supporting  this  COI. 

3.2.2.  MOPs  and  Performance  Criteria: 

3.2.2. 1.  MOP  2-1:  number  of  real-time  computer  processors  that  cannot  be  restored  using 
the  ethernet.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

3.2. 2. 2.  MOP  2-2:  number  of  workstations  that  cannot  be  restored  to  full  mission 
capability  from  other  nodes  over  the  ethernet.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

3. 2.2.3.  MOP  2-3:  number  of  computers  that  cannot  be  restored  to  full  mission  capability 
using  the  tape  back-ups.  (CRITICAL) 

Performance  Criterion:  zero  detected. 
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3. 2. 2.4.  MOP  2-4:  number  of  interfaces  not  achieved  between  the  computers  and 
equipment.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

3.2.2.5.  MOP  2-5:  number  of  menu  items  that  cannot  be  selected  by  a  single  keystroke  or 
a  single  combination  of  two  keystrokes  (as  in  alt/F10).  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.2.2.6.  MOP  2-6:  number  of  mission  data  segment  changes  not  executable  via  operator 
menu  selection. 

Performance  Criterion:  zero  detected. 

3.2. 2.7.  MOP  2-7:  number  of  equipment  checks  and  alignments  not  executable  via 
keyboard  entry. 

Performance  Criterion:  zero  detected. 

3. 2.2.8.  MOP  2-8:  percent  of  selected  text  files  adequately  printed  to  the  LN05R  printers. 
(SIGNIFICANT) 

Performance  Criterion:  91%. 

3.2.2.9.  MOP  2-9:  percent  of  selected  graphics  files  adequately  printed  to  the  LN05R 
printers.  (SIGNIFICANT) 

Performance  Criterion:  91%. 

3.2.2.10.  MOP  2-10:  percent  of  screen  images  adequately  printed  to  the  LN05R  printers. 
(SIGNIFICANT) 

Performance  Criterion:  91%. 

3.2.2. 1 1.  MOP  2-11:  number  of  inadequate  plots  from  the  SFIR.  (SIGNIFICANT) 
Performance  Criterion:  zero  detected. 

3.2.2.12.  MOP  2-12:  number  of  Digital  Command  Language  (DCL)  level  functions  that 
cannot  be  performed  by  the  operators. 

Performance  Criterion:  zero  detected. 

3.2.2.13.  MOP  2-13:  number  of  nodes  that  cannot  be  assigned  or  reassigned  without 
bringing  the  system  down.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

3.2.2.14.  MOP  2-14:  uptime  ratio  (UTR)  of  the  RSTN  computer  system. 

(SIGNIFICANT) 

Performance  C'iterion:  a  minimum  of  98 %.  This  is  based  on  the  requirement  for  no 
degradation  to  the  documented  UTR  of  the  existing  computer  system 
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3.2.2.15.  MOP  2-15:  UTR  of  the  SOON  computer  system.  (SIGNIFICANT! 
Performance  Criterion:  a  minimum  of  98‘T.  This  is  based  on  the  requirement  for  no 
degradation  to  the  documented  UTR  of  the  existing  computer  system. 

3.2.2.16.  MOP  2-16:  time  required  for  operators  to  load  and  run  new  version  software. 
(CRITICAL) 

Performance  Criterion:  less  than  X  hours. 

3.2.2.17.  MOP  2-17:  performs  required  functions  where  the  computer  individually  reads, 
writes,  interacts,  and  controls  for  each  lock  in  amplifier  (LIA).  (SIGNIFICANT) 
Performance  Criterion:  adequate. 

3.2.2.18.  MOP  2-18:  percent  of  faults  detected  by  the  automatic  monitoring  system. 
(SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  99 7r. 

3.2.2.19.  MOP  2-19:  false  alarm  rate  of  faults  by  the  automatic  monitoring  system. 
(SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1  %. 

3.2.2.20.  MOP  2-20:  number  of  incorrect  status  indications.  (SIGNIFICANT) 
Performance  Criterion:  zero  detected. 

3.2.2.21.  MOP  2-21:  number  of  work  stations  assigned  as  .system  console  that  cannot 
perform  calibration  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

3.2.2.22.  MOP  2-22:  number  of  work  stations  assigned  as  system  console  that  cannot 
control  the  data  image  collection  function  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

3.2.2.23.  MOP  2-23:  number  of  work  stations  assigned  as  system  console  that  cannot 
control  the  scheduling  of  movie  replay  function  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

3.2.2.24.  MOP  2-24:  number  of  work  stations  assigned  as  system  console  that  cannot 
control  the  remote  shutdown  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

3.2.2.26.  MOP  2-25:  number  of  inadequate  magnetic  contour  and  Stoneyhurst  plots 
performed  by  the  system  using  data  provisioned  from  the  telescopes  under  control  of  the 
workstations.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 
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3.2.3.  Mission  Scenarios.  Technicians  and  analysts  will  document  any  computer-related 
anomalies  that  occur  during  normal  operations  and  maintenance  of  the  SEON  computers. 
SEON  analysts  will  attempt  to  change  mission  data  segments  using  keyboard  entries. 
Technicians  will  perform  equipment  checks  and  alignments  using  workstation  keyboard 
entries.  To  evaluate  restoring  computer  information,  all  data  on  the  computer  under  test 
will  first  be  saved  to  the  restoring  device  and  then  erased.  Technicians  will  then  attempt  to 
restore  the  data  to  the  computers.  Restored  data  will  be  verified  to  ensure  no  errors  or  hist 
data  occurs.  The  interface  between  the  new  computer  and  the  LlAs  will  be  tested  by 
technicians  during  normal  preventive  maintenance  inspections.  To  test  the  BIT.  status 
indications  will  be  monitored.  Status  indications  that  do  not  occur  spontaneously  will  be 
induced  using  locally  developed  checklists.  Solar  data  gathered  by  the  SOON  will  be  sent 
to  the  DIPS  and  manipulated  to  evaluate  the  effect  of  the  upgrade  on  DIPS  operations. 

3.2.4.  Method  of  Evaluation.  All  data  pertaining  to  the  new  computers  will  be  compared 
to  the  performance  criteria.  The  UTR  of  the  RSTN  and  SOON  computer  systems  will  be 
calculated  separately  using  the  formula  below.  A  computer  system  will  be  considered 
"down,"  if  it  causes  a  significant  outage  as  defined  in  Air  Force  Space  Forecast  Center 
Regulation  (AFSFCR)  105-1,  Attachment  4  (refer  to  Figure  3- 1 ).  Qualified  analysts  and 
programmers  will  use  their  professional  judgement  to  rate  the  computer  enhanced  data  as 
either  "adequate"  or  "not  adequate."  All  critical  MOPs  must  meet  user  requirements  in 
order  to  rate  this  COI  as  "resolved  satisfactory."  In  addition,  MOPs  marked  as  "significant” 
were  not  deemed  "critical,"  but  were  determined  to  have  a  significant  impact  on  mission 
accomplishment.  To  cause  the  COI  to  be  rated  "resolved  unsatisfactory,"  the  results  of 
these  MOPs  must  fail  to  meet  user  criteria  and  prevent  mission  accomplishment. 

UTR  =  TOTAL  OPERATING  HOURS  MINUS  TOTAL  DOWNTIME  HOURS 

TOTAL  OPERATING  HOURS 

3.3.  COI-3:  Does  the  upgraded  RSTN  allow  detection  of  solar  events? 

3.3.1.  Scope.  The  Swept  Frequency  Interferometric  Radiometer  (SFIR)  and 

discrete  frequency  radiometer  will  be  evaluated  during  normal  operation  and  maintenance 
each  day  during  the  90-day  QOT&E.  MOP  3-12  is  the  critical  MOP  supporting  this  COI. 

3.3.2.  MOPs  and  Performance  Criteria: 


3.3.2. 1 .  MOP  3-1:  percent  of  solar  bursts  detected  by  the  current  SFIR  between 
30-75  megahertz  (MHz)  that  are  also  detected  by  the  new  SFIR  between  30-75  MHz  while 
both  SFIRs  are  operational.  (SIGNIFICANT) 

Performance  Criterion:  97Vr. 


3.3. 2. 2.  MOP  3-2:  percent  of  solar  bursts  detected  by  other  SFIRs  (at  other  SEON  sites) 
between  30-75  MHz  that  are  also  detected  by  the  new  SFIR.  (SIGNIFICANT) 

Performance  Criterion:  95  Vr  of  bursts  detected  by  other  sites  during  concurrent  patrol 
periods. 

3.3.2. 3.  MOP  3-3:  subjective  rating  by  qualified  analysts  of  solar  sweep  signatures 
uetected  between  30-75  MHz.  (.SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.3. 2.4.  MOP  3-4:  subjective  rating  by  qualified  analysts  of  solar  sweep  intensities 
between  30-75  MHz.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.3.2.5.  MOP  3-5:  speed  of  Type  2  shocks.  (SIGNIFICANT) 

Performance  Criterion:  within  500  kilometers  per  second  of  the  speed  indicated  by  the 
current  system. 

3.3. 2.6.  MOP  3-6:  minimum  sensitivity  of  the  new  SFIR  at  30  MHz.  (SIGNIFICANT) 
Performance  Criterion:  10  Solar  Flux  Units  (SFUs). 

3.3. 2.7.  MOP  3-7:  minimum  sensitivity  of  the  new  SFIR  at  250  MHz.  (SIGNIFICANT) 
Performance  Criterion:  40  SFUs. 

3.3.2.X.  MOP  3-X:  number  of  solar  events,  types  of  solar  burst  signatures ,  and  burst 
intensities  between  75-250  MHz  detected  by  the  SFIR.  There  is  no  performance  criterion 
for  this  MOP. 

3.3. 2.9.  MOP  3-9:  percent  of  solar  intensity  difference  between  that  indicated  by  the 
discrete  frequency  radiometer  computer  and  the  charted  intensity.  (SIGNIFICANT) 
Performance  Criterion:  10%  or  less. 

3.3.2.10.  MOP  3-10:  the  computer  reported  burst  characteristics  (start  time,  maximum 
time,  stop  time,  mean  flux,  integrated  flux,  and  peak  flux)  compared  to  the  charted 
characteristics.  (SIGNIFICANT) 

Performance  Criteria:  plus  or  minus  5  seconds  for  times  and  within  10%  for  mean  flux, 
integrated  flux,  and  peak  flux. 

3.3.2. 1 1.  MOP  3-11:  number  of  solar  bursts  the  discrete  frequency  radiometer  incorrectly 
identifies  (excluding  RFI).  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.3.2.12.  MOP  3-12:  subjective  rating  by  experienced  RSTN  analysts  and  programmers  of 
discrete  frequency  bursts.  (CRITICAL) 

Performance  Criterion:  adequate. 
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3.3.3.  Mission  Scenarios.  Data  will  consist  of  the  solar  information  collected  by  the 
discrete  frequency  radiometer,  the  new  SFIR.  the  existing  SFIR,  other  SFlks  in  the  SEON, 
and  Culgoora  Solar  Observatory  in  Australia.  Simulated  data  will  be  used  if  spontaneous 
solar  events  do  not  occur.  Technicians  will  test  the  SFIR  and  automated  discrete 
frequency  radiometer  a  minimum  of  30  times  for  proper  performance  as  a  part  of  their 
normal  preventative  maintenance.  Technicians  will  test  the  SFIR's  sensitivity  during 
preventative  maintenance  inspections. 

3.3.4.  Method  of  Evaluation.  The  solar  data  between  30-75  MHz  from  the  new  SFIR  will 
be  compared  to  the  solar  data  between  30-75  MHz  from  the  existing  SFIR  and  other  SFIRs 
in  the  SEON.  The  solar  data  gathered  from  the  new  SFIR  and  discrete  frequency 
radiometer  will  also  be  compared  to  the  solar  data  from  Culgoora  Solar  Observatory  in 
Australia.  Qualified  analysts  will  use  their  professional  judgement  to  make  subjective 
ratings  on  nonquantifiable  data.  Quantified  RSTN  data  will  be  compared  to  the 
performance  criteria.  MOP  3-12  was  determined  to  be  critical  to  mission  accomplishment 
and  must  meet  user  criteria  to  rate  the  COI  as  "resolved  satisfactory."  In  addition,  MOPs 
marked  as  "significant"  were  determined  to  have  a  significant  impact  on  mission 
accomplishment.  To  cause  the  COI  to  be  rated  "resolved  unsatisfactory,"  the  results  of 
these  MOPs  must  fail  to  meet  user  criteria  and  prevent  mission  accomplishment. 

3.4.  CO  1-4:  Does  the  upgraded  SOON  allow  detection  of  solar  events? 

3.4.1.  Scope.  The  test  team  will  evaluate  CCD  images,  the  Intelligent  Data  Acquisition 
System  (IDAS),  and  the  DIPS  capabilities  during  normal  operation  and  maintenance  each 
day  of  the  QOT&E.  MOPs  4-16,  and  4-32  are  the  critical  MOPs  supporting  this  COI. 

3.4.2.  MOPs  and  Performance  Criteria: 

3.4.2. 1.  MOP  4-1:  subjective  rating  by  qualified  analysts  of  H-Alpha  images. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.2.  MOP  4-2:  minimum  resolution  of  large  scale  H-AIpha  images. 

(SIGNIFICANT) 

Performance  Criterion:  2.2  arc  seconds. 

3.4. 2.3.  MOP  4-3:  minimum  resolution  of  full  disk  H-Alpha  images. 

(SIGNIFICANT) 

Performance  Criterion:  12.5  arc  seconds. 

3.4. 2.4.  MOP  4-4:  subjective  rating  by  qualified  analysts  of  spectrograph ic  images. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 


3 .4.2.5.  MOP  4-5:  minimum  resolution  of  large  scale  spectrographic  images. 
(SIGNIFICANT) 

Performance  Criterion:  2.2  arc  seconds. 

3.4.2. 6.  MOP  4-6:  minimum  resolution  of  full  disk  spectrographic  images. 
(SIGNIFICANT) 

Performance  Criterion:  12.5  arc  seconds. 

3.4. 2.7.  MOP  4-7:  subjective  rating  by  qualified  analysts  of  H-Alpha  images  transferred  to 
the  DIPS  from  the  RS170  output  of  the  CCD.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.S.  MOP  4-8:  subjective  rating  by  qualified  analysts  of  spectrographic  magnetic  data 
transferred  to  the  DIPS. 

Performance  Criterion:  adequate. 

3.4. 2.9.  MOP  4-9:  number  of  frames  transferred  to  the  DIPS  from  the  SOON. 
(SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  12  frames  per  minute. 

3.4.2.10.  MOP  4-10:  DIPS  playback  speed. 

Performance  Criteria:  up  to  30  frames  per  second. 

3.4.2. 1 1.  MOP  4-11:  number  of  frames  that  can  be  stored  by  the  DIPS. 

Performance  Criterion:  a  minimum  of  800  frames. 

3.4.2.12.  MOP  4-12:  resolution  of  each  frame  (stored  in  the  DIPS). 

Performance  Criterion:  a  minimum  of  512  X  512  X  8  bits  per  frame. 

3.4.2.13.  MOP  4-13:  sufficiency  of  shades  of  gray  (of  stored  DIPS  images). 

Performance  Criterion:  256  gray  shades  to  allow  the  capability  for  coronal  hole  detection 
at  10830  angstroms. 

3.4.2.14.  MOP  4-14:  speed  of  overlaying  pictures  on  stored  grids  (in  the  DIPS). 
Performance  Criterion:  a  minimum  of  one  frame  per  second. 

3.4.2.15.  MOP  4-15:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
histogram  capability.  (CRITICAL) 

Performance  Criterion:  adequate. 

3.4.2.16.  MOP  4-16:  minimum  resolution  of  H-Alpha  and  visible  light  (including  Mg  B:). 
(SIGNIFICANT) 

Performance  Criteria:  2.2.  arc  seconds  for  large  scale.  12.5  for  full  disk. 


3-18 


3.4.2.17.  MOP  4-17:  resolution  of  the  detected  solar  magnetic  field.  (SIGNIFICANT) 
Performance  Criterion:  a  minimum  of  5  arc  seconds  at  large  scale. 


3.4.2. IS.  MOP  4- IS:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
image  enhancement  of  gray  scale  and  pseudo  color.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.19.  MOP  4-19:  rate  of  optical  scan  conversion.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.20.  MOP  4-20:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
adequacy  of  stored  images  compared  to  the  live  images.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.21.  MOP  4-21:  subjective  rating  by  trained  NGDC  analysts  of  the  adequacy  of 
archived  solar  data. 

Performance  Criterion:  adequate. 


3.4.2.22.  MOP  4-22:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
spectrographic  output  data.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 


3.4.2.23.  MOP  4-23:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
digital  images,  (scaled,  standardized,  and  superimposed).  (SIGNIFICANT) 

Performance  Criterion:  adequate. 


3.4.2.24.  MOP  4-24:  amount  of  degradation  in  current  provided  operability. 
(SIGNIFICANT) 

Performance  Criterion:  no  degradation. 


3.4.2.25.  MOP  4-25:  performs  required  analog  to  digital  (A/D)  converter  functions. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.26.  MOP  4-26:  number  of  SOON  significant  outages  caused  by  an  A/D  converter. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected.  A  significant  outage  is  defined  in  AFSFCR  105-1, 
Attachment  4  (refer  to  Figure  3-1). 

3.4.2.27.  MOP  4-27:  number  of  A/D  converter  failures.  (.SIGNIFICANT) 

Performance  Criterion:  zero  detected.  This  is  based  on  the  requirement  for  18  months  of 
operation  without  a  failure. 
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3.4.2.2X.  MOP  4-2X:  performs  required  digital  to  analog  (D/A)  converter  functions. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.4.2.29.  MOP  4-29:  number  of  SOON  significant  outages  caused  by  a  D/A  converter. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected.  A  significant  outage  is  defined  in  AFSFCR  105- 1 , 
Attachment  4  (refer  to  Figure  3-1). 

3.4.2.30.  MOP  4-30:  number  of  D/A  converter  failures.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.4.2.31.  MOP  4-31:  subjective  rating  by  experienced  SOON  analysts  and  programmers  of 
solar  flare  analysis  function.  (CRITICAL) 

Performance  Criterion:  adequate. 

3.4.3.  Mission  Scenarios.  All  system  failures  will  be  documented  using  established 
procedures  and  the  QOT&E  Operations  Record.  Technicians  will  test  the  digital  to  analog 
(D/A)  and  analog  to  digital  (A/D)  converters  and  report  their  findings  using  Air  Force 
Technical  Order  (AFTO)  Forms  349  or  equivalent  and  the  QOT&E  Maintenance  Record. 
Qualified  analysts  and  programmers  will  collect  a  minimum  of  30  solar  images  from  the 
various  SEON  interfaces.  Technicians  will  test  each  CCD  and  IDAS  A/D  and  D/A 
converter  a  minimum  of  30  times.  To  test  the  CCDs,  live  solar  images  and  simulated  solar 
images  will  be  used.  To  simulate  solar  images,  a  calibrated  test  target  will  be  placed  in  the 
light  path  of  the  telescope  to  project  images  to  the  CCDs.  To  test  the  adequacy  of  images 
transferred  to  the  DIPS,  prerecorded,  test,  and  live  solar  images  will  be  transferred  from  the 
computers  and  CCDs. 

3.4.4.  Method  of  Evaluation.  Qualified  analysts  will  use  their  professional  judgement  to 
make  subjective  ratings  on  nonquantifiable  data.  They  will  rate  this  data  as  either 
"adequate"  or  "not  adequate."  Quantifiable  data  will  be  compared  to  the  performance 
criteria.  All  critical  MOPs  must  meet  user  requirements  in  order  to  rate  this  COI  as 
"resolved  satisfactory."  In  addition,  MOPs  marked  as  "significant"  were  determined  to 
have  a  significant  impact  on  mission  accomplishment.  To  cause  the  COI  to  be  rated 
"resolved  unsatisfactory,"  the  results  of  these  MOPs  must  fail  to  meet  user  criteria  and 
prevent  mission  accomplishment. 

3.5.  C'OI-5:  Is  the  SEON  software  supportable? 

3.5.1.  Scope.  Testing  for  this  COI  is  divided  into  two  sections.  The  first  section,  the 
capability  of  the  programmers  to  perform  their  mission,  will  be  evaluated  during  normal 
operation  and  maintenance  of  the  SEON  software.  The  second  section,  software  testing, 
will  be  evaluated  by  the  Software  Test  Manager  (STM).  Software  testing  will  focus  on 
four  areas:  software  maintainability,  usability,  support  resources,  and  maturity.  Software 
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four  areas:  software  maintainability,  usability,  support  resources,  and  maturity.  Software 
maintainability  examines  characteristics  of  the  software  source  code  and  documentation 
which  affect  the  ability  of  programmers  and  analysts  to  modify  and  update  the  software. 
Software  usability  examines  attributes  which  define  the  "ease-of-use"  of  a  system. 

Software  support  resources  evaluation  examines  personnel,  training,  support  systems,  and 
facilities.  Software  maturity  measures  the  software’s  progress  in  its  evolution  toward 
satisfaction  of  all  documented  user  requirements.  MOP  5-1  is  the  critical  MOP  supporting 
this  COl. 

3.5.2.  MOPs  and  Performance  Criteria: 

3.5.2. 1.  MOP  5-1:  usability  of  the  SEON  software.  (CRITICAL) 

Performance  Criterion:  Subjective. 
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AFSFCR  105-1  A4-1 

ATTACHMENT  4 

SEON  EQUIPMENT  OUTAGE  IMPACTS 

EQUIPMENT 

IMPACT 

Radio  (RIMS): 

An  outege  la  SIGNIFICANT  If  data  cannot  be  gathered  or  analyzed  on: 

-246  MHz 
-2695  MHz 

-  6600  MHz 

-  Two  or  more  adjacent  frequence* 

-  Three  or  more  frequenciee 

Radio  (SFIR): 

An  outogo  is  SIGNIFICANT  It  swoop  ttms  or  typo  eonnot  bo  dotermlnod. 

Optical  (SOON): 

An  outogo  Is  SIONOTCANT  H  ths  SOON  syststn  (which  Indudos  DIPS)  esnnet: 

-  Be  calibrated 

-  Detect  or  analyze  flares  In  automatic  mode 

-  Analyze  real-time  H-alpha  or  White  light  images 

-  Produce  magnetogram  a 

•  Track  the  eun  automatically 

SEON  Computer  Systom: 

An  outogo  Is  SIGNIFICANT  It  Any  of  ths  sbovo  llslod  SOON  or  RSTN  oporsttons  eonnot 
bo  dons  in  outomsdc  mode. 

Communications: 

A  teletype  outege  it  SIGNIFICANT  if  either  send  fit  receive  capability  ia 
completely  lost  at  the  atta. 

A  DSN  {previously  known  ee  autovon)  or  commercial  phone  outege  le  SIGNIFICANT  if 
the  teletype  circuit  is  also  down,  leaving  no  functioning  moans  of  oommunicationa. 

NOTES: 

1.  Ail  SEON  equipment,  computer,  or  communication*  outages  not  specified  above  aa 

SIGNIFICANT  are  MINIMAL,  unless  declared  SIGNIFICANT  by  the  detachment  commander, 

AF3FC/DCM,  or  AFSFC/DOO. 

2.  An  outage  It  normally  MINIMAL  If  date  Is  degraded  due  to  minor  equipment  problems,  but  can 
•till  be  collected  and  analyzed. 

Figure 

3-1.  SEON  Equipment  Outage  Impacts 

3. 5. 2. 2.  MOP  3-2:  maturity  of  the  SEON  software.  (SIGNIFICANT) 

Performance  Criterion:  Subjective. 

3. 5. 2.3.  MOP  5-3:  maintainability  of  the  SEON  software.  (SIGNIFICANT) 
Performance  Criterion:  subjective. 

3.5. 2.4.  MOP  5-4:  support  resources  of  the  SEON  software.  (SIGNIFICANT) 
Performance  Criterion:  subjective. 

3. 5. 2.5.  MOP  5-5:  life-cycle  process  of  the  SEON  software.  (SIGNIFICANT) 
Performance  Criterion:  subjective. 

3.5. 2.6.  MOP  5-6:  time  required  to  update  and  install  completely  new  source  software 
distribution  into  the  system.  (SIGNIFICANT) 

Performance  Criterion:  less  than  X  hours. 

3.5. 2.7.  MOP  5-7:  number  of  approved  software  changes  the  programmers  cannot 
introduce  into  the  application  software.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.5.2. X.  MOP  5-X:  system  impact  of  testing  approved  software  changes. 

Performance  Criterion:  subjective. 

3.5.2.9.  MOP  5-9:  number  of  tested  software  changes  that  cannot  be  introduced  into  the 
system  without  rebooting  the  computer. 

Performance  Criterion:  zero  detected. 

3.5.2.10.  MOP  5-10:  number  of  software  development  tools  the  programmers  cannot 
effectively  use.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.5.2. 1 1.  MOP  5-11:  number  of  program  documentation  items  the  programmers  cannot 
update.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.5.2.12.  MOP  5-12:  number  of  programming  tasks  that  cannot  be  performed  due  to 
inadequate  training.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.5.3.  Mission  Scenarios.  The  SEON  programmers  will  document  all  actions  they  take 
during  the  90-day  QOT&E  in  their  QOT&E  Operations  Record.  They  will  pay  particular 
attention  to  the  time  required  to  update  and  install  the  completely  new  source  software 
distribution  into  the  system.  This  time  will  include  loading  the  software  into  the  system, 
compiling  and  linking,  distributing  the  software  to  other  nodes  over  the  Local  Area 
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ATTACHMENT  « 

TIMELINESS  CRITERIA 

1.  AUTOMATIC  OPERATIONS:  (transmit  as  soon  as  possible.  but  no 

Lto  exceed  these  limits.) 

CODE  TYPE 

EVENT  LEVEL 

NON-EVENT  LEVEL 

EVENT 

2  min  after  event  atart 

Not  Applicable 

(5  min  for  e  SITEC) 

FLARE:  preliminary 

5  min  after  max 

Not  Requlr  ad 

final 

10  min  altar  and 

1 5  min  after  end 

OALAS:  preliminary 

10  min  after  event  atart 

30  min  after  stert 

final 

20  mm  after  end 

1 5  min  after  end 

BURST:  preliminary 

5  min  after  max  (or  1st  NSM  event  peak) 

Not  Required  (30  mm  after  start  for  NSM) 

final 

10  mtn  altar  end  (45  min  tor  a  NSM) 

30  min  after  end  (45  min  for  NSM) 

SWEEP:  prallmlnary 

10  min  after  type  Identification 

Not  Required  (30  min  after  start  data 

availability  for  a  Code  6,  7,  or  8) 

final 

30  min  after  data  availability 

45  min  after  end  data  ave'isbiiity 

2.  SEMIAUTOMATIC  OR 

MANUAL  OPERATIONS:  (transmit  as  soon  as  oossibla.  but  not  to  exceed  these  limits.) 

CODE  TYPE 

EVENT  LEVEL 

EVENT 

5  min  after  event  start 

Not  Applicable 

FLARE:  preliminary 

15  min  after  max 

Not  Required 

final 

20  min  after  end 

30  min  after  end 

OALAS:  preliminary 

— . . (same  as  for  automatic  operation*) - - - 

final 

— . . — -  (same  as  for  automatic  operations)  . — . 

BURST:  prallmlnary 

20  min  altar  max  (or  1st  NSM  event  peak) 

Not  Required  (30  mm  after  start  for  NSM) 

final 

30  min  after  and  (45  min  for  a  NSM) 

45  min  after  end 

SWEEP:  prallmlnary 

-  (same  as  tor  automatic  operations)  - - 

final 

.  (same  aa  for  automatic  operations)  . 

NOTE:  NSM  -  Noise  Storm. 

Figure  3-2.  Timeliness  C  riteria. 

Network  (LAN),  and  initiating  (running)  the  software.  The  STM  is  responsible  for 
conducting  the  AFOTEC -designed  software  evaluation  in  accordance  with  the  AFOTEC 
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XOO-Series  pamphlets.  The  STM  will  conduct  software  evaluations  throughout  our 
involvement  of  the  system’s  development  and  test.  These  evaluations  may  take  place  at 
several  locations:  software  facilities  of  the  contractor,  agency  responsible  for  maintaining 
the  software  (SM-ALC),  AFSFC.  and  at  Det  1,  AFOTEC.  The  STM  will  ensure  the  guides 
and  questionnaires  from  the  AFOTEC  XOO-Series  pamphlets  are  completed  in  compliance 
with  current  guidance. 

3.5.4.  Method  of  Evaluation.  To  evaluate  the  ability  of  the  programmers  to  perform  their 
mission,  all  data  relating  to  programming  functions  will  be  summarized  and  compared  to 
the  performance  criteria.  For  the  software  evaluations,  the  STM  will  collect  and  process 
data  from  the  four  software  areas.  Each  of  the  guides  and  questionnaires  will  result  in 
qualitative  and  quantitative  information  on  an  attribute  of  the  software.  MOP  5- 1  was 
determined  to  be  critical  to  mission  accomplishment  and  must  meet  user  criteria  to  rate  the 
COl  as  "resolved  satisfactory.”  In  addition,  MOPs  marked  as  "significant"  were  determined 
to  have  a  significant  impact  on  mission  accomplishment.  To  cause  the  COI  to  be  rated 
"resolved  unsatisfactory,"  the  results  of  these  MOPs  must  fail  to  meet  user  criteria  and 
prevent  mission  accomplishment. 

3.6.  COI-6:  Does  the  SEON  adequately  report  solar  events? 

3.6.1.  Scope.  The  test  team  will  test  the  ability  of  the  SEON  to  transmit  all  required 
messages  over  the  AWN.  MOPs  6-2,  and  6-4  are  the  critical  MOPs  supporting  this  COI. 

3.6.2.  MOPs  and  Performance  Criteria. 

3.6.2. 1.  MOP  6-1:  percent  of  message  formats  transmitted  1  AW  Air  Force  Space  Forecast 
Center  Pamphlet  (AFSFCP)  105-5.  (SIGNIFICANT) 

Performance  Criterion:  97%  (refer  to  Table  3-1). 

3.6. 2. 2.  MOP  6-2:  time  between  solar  activity  and  transmission  of  the 
required  message.  (CRITICAL) 

Performance  Criterion:  IAW  AFSFCR  105-1.  Attachment  4  (refer  to  Figure  3-2). 

3. 6.2. 3.  MOP  6-3:  number  of  errors  between  transmitted  and  received  messages. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3. 6.2. 4.  MOP  6-4:  number  of  solar  events  the  analysts  are  unable  to  respond  to  within 
time  constraints  listed  in  Air  Force  Space  Command  Regulation  (AFSCR)  105-1. 
(CRITICAL) 

Performance  Criterion:  zero  detected. 

3.6.3.  Mission  Scenarios.  A  copy  of  all  messages  transmitted  by  the  Palehua  Solar 
Observatory  during  the  QOT&E  will  be  given  to  the  Test  Director.  A  copy  of  all  messages 
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received  by  the  Air  Force  Space  Forecast  Center  (AFSFC)  from  the  Palehua  Solar 
Observatory  will  be  sent  to  the  Test  Director.  If  additional  messages  are  required  to 
statistically  support  the  findings,  messages  clearly  marked  "PALEHUA  SEON  TEST 
MESSAGE"  will  be  transmitted. 

3.6.4.  Method  of  Evaluation.  All  data  gathered  during  this  event  will  be  summarized  and 
compared  to  the  performance  criteria.  All  critical  MOPs  must  meet  user  requirements  in 
order  to  rate  this  COI  as  "resolved  satisfactory."  in  addition,  MOPs  6-1  and  6-3  marked 
as  "significant"  were  not  deemed  "critical,"  but  were  determined  to  have  a  significant 
impact  on  mission  accomplishment.  To  cause  the  COI  to  be  rated  "resolved 
unsatisfactory,"  the  results  of  these  MOPs  must  fail  to  meet  user  criteria  and  prevent 
mission  accomplishment. 

3.7.  COI-7:  Is  the  SEON  supportable? 

3.7.1.  Scope.  The  data  supporting  logistics  supportability,  readiness,  maintainability, 
training,  and  sufficiency  of  T.O.s  will  be  collected  during  normal  operations  and 
maintenance  of  the  SEON.  MOP  7-27  is  the  critical  MOP  supporting  this  COI. 

3.7.2.  MOPs  and  Performance  Criteria: 

3.7.2. 1.  MOP  7-1:  mean  time  between  critical  failure  (MTBCF)  of  the  RSTN. 
(SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  750  hours. 

3.7 .2.2.  MOP  7-2:  MTBCF  of  the  SOON.  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  750  hours. 

3.1.23.  MOP  7-3:  mean  time  between  maintenance  corrective  (MTBMc). 
(SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  200  hours. 

3.7.2.4.  MOP  7-4:  mean  repair  time  (MRT)  of  the  SEON.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  4  hours. 

3.7 .2.5.  MOP  7-5:  MRT  of  the  embedded  computer  resources.  (SIGNIFICANT) 
Performance  Criterion:  a  maximum  of  1  hour. 

3.7.2.6.  MOP  7-6:  MRT  of  the  SFIR.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1.5  hours. 

3.7. 2.7.  MOP  7-7:  Max  time  to  repair  (Max  TTR)  for  the  SFIR.  (SIGNIFICANT) 
Performance  Criterion:  a  maximum  of  X  hours. 
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3.7.2.X.  MOP  7-X:  Max  TTR  for  the  CCD.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1  hour. 

3.7.2.9.  MOP  7-9:  Max  TTR  for  the  UPS.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1  hour. 

3.7.2.10.  MOP  7-10:  Max  TTR  for  the  S EON.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  3  hours  for  95%  of  all  failures. 

3.7.2. 1 1.  MOP  7-11:  number  of  failures  not  detected  through  test  points  using  full 
resources.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.7.2.12.  MOP  7-12:  percent  of  detected  faults  corrected  by  replacing  one  Line 
Replaceable  Unit  (LRU).  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  90‘/r. 

3.7.2.13.  MOP  7-13:  percent  of  detected  faults  corrected  by  replacing  a  maximum  of  2 
LRUs.  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  95%. 

3.7.2.14.  MOP  7-14:  percent  of  detected  faults  corrected  by  replacing  a  maximum  of  3 
LRUs.  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  99%. 

3.7.2.15.  MOP  7-15:  number  of  special  tools  required  to  maintain  the  SEON. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.7.2.16.  MOP  7-16:  number  of  peculiar  test  equipment  required  to  maintain  the  SEON. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.7.2.17.  MOP  7-17:  weekly  preventative  maintenance  inspection  (PM1)  times  for 
maintaining  the  upgraded  portions  of  the  SEON.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  30  minutes. 

3.7. 2. IX.  MOP  7-1X:  weekly  PMI  times  for  maintaining  the  upgraded  portions  of  the 
RSTN.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  30  minutes. 

3.7.2.19.  MOP  7-19:  amount  of  increased  time  required  to  perform  weekly  PMI  for 
maintaining  the  non-upgraded  portions  of  the  RSTN.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 
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3.7.2.20.  MOP  7-20:  amount  of  increased  time  required  to  perform  weekly  PM1  for 
maintaining  the  non-upgraded  portions  of  the  SOON.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.7.2.21.  MOP  7-21:  number  and  skill  level  of  personnel  required  to  perform  any  SEON 
maintenance  routine.  (SIGNIFICANT) 

Performance  Criterion:  one  30452  technician. 

3.7.2.22.  MOP  7-22:  number  of  failures  requiring  more  than  removal  and  replacement  of 
defective  LRUs  on  the  new  SFIR,  CCDs,  IDAS,  or  computer  resource.  (SIGNIFICANT) 
Performance  Criterion:  zero  detected. 

3.7.2.23.  MOP  7-23:  depot-level  task  completion  time.  (SIGNIFICANT) 

Performance  Criterion:  Less  than  6  hours. 

3.7.2.24.  MOP  7-24:  MTBCF  of  the  SFIR  and  discrete  frequency  radiometer.  There  is 
no  performance  criteria  for  this  MOP. 

3.7.2.25.  MOP  7-25:  percent  of  initial  spares  support  listing  (ISSL)  items  filled  at  site 
prior  to  QOT&E. 

Performance  Criterion:  a  minimum  of  X()%. 

3.7.2.20.  MOP  7-26:  percent  of  critical  ISSL  items  filled  at  site  prior  to  QOT&E. 
(SIGNIFICANT) 

Performance  Criterion:  100%. 

3.7.2.27.  MOP  7-27:  number  of  significant  outages  caused  by  erroneous  technical  data. 
(CRITICAL) 

Performance  Criterion:  zero  detected. 

3.7.2.2X.  MOP  7-2K:  subjective  assessment  by  qualified  programmers,  technicians,  and 
analysts  of  the  technical  data.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.7.2.29.  MOP  7-29:  number  of  significant  outages  caused  by  inadequate  training. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

3.7.2.30.  MOP  7-30:  subjective  assessment  by  qualified  SEON  operators,  maintainers. 
and  analysts  of  the  training.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 


3.7.2.31.  MOP  7-31:  number  of  discrepancies  between  maintenance  AFSC  job 
descriptions  and  the  tasks  performed. 

Performance  Criterion:  zero  detected. 


3.7.2.32.  MOP  7-32:  adequacy  of  the  course  chart  for  maintenance.  (SIGNIFICANT) 
Performance  Criterion:  adequate. 

3.7.2.33.  MOP  7-33:  adequacy  of  plan  of  instruction  for  maintenance.  (SIGNIFICANT) 
Performance  Criterion:  adequate. 

3.7.2.34.  MOP  7-34:  adequacy  of  the  provided  student  maintenance  training  material. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.7.2.35.  MOP  7-35:  adequacy  of  the  maintenance  training  program  and  training 
equipment  plan.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.7.2.36.  MOP  7-36:  number  of  discrepancies  between  operations  AFSC  job  descriptions 
and  the  tasks  performed. 

Performance  Criterion:  zero  detected. 


3.7.2.37.  MOP  7-37:  adequacy  of  the  course  chart  for  operations. 
Performance  Criterion:  adequate. 


(SIGNIFICANT) 


3.7.2.38.  MOP  7-38:  adequacy  of  plan  of  instruction  for  operations.  (SIGNIFICANT) 
Performance  Criterion:  adequate. 


3.7.2.39.  MOP  7-39:  adequacy  of  the  provided  student  operations  training  material. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 


3.7.2.40.  MOP  7-40:  adequacy  of  the  operations  training  program  and  training  equipment 
plan.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.7.2.41.  MOP  7-41:  UTR  of  the  SOON  system. 

Performance  Criterion:  a  minimum  of  97%. 

3.7.2.42.  MOP  7-42:  UTR  of  the  RSTN  system. 

Performance  Criterion:  a  minimum  of  97%. 


3.7.2.43.  MOP  7-43:  UTR  of  the  SEON  system. 
Performance  Criterion:  a  minimum  of  97%. 


3.7.3.  Mission  Scenarios.  During  normal  operations,  all  system  problems  will  be 
documented  using  established  procedures  and  the  QOT&E  Operations  Records  and  QOT&E 
Maintenance  Records.  Copies  of  all  documentation  (Master  Station  Logs,  Equipment 
Outage  Reports,  AFTO  Forms  349  or  equivalent,  etc.)  will  be  given  to  the  Test  Director. 
Induced  failures  will  be  used  by  the  test  team  to  ensure  that  all  system  areas  are  tested  for 
maintainability.  The  Test  Director  will  ensure  documents  are  marked  if  the  data  was 
gathered  during  normal  operations  or  during  a  QOT&E  induced  scenario.  An  inventory  of 
the  ISSL  will  be  conducted  at  the  beginning  and  end  of  the  QOT&E,  During  normal 
operations  and  maintenance,  any  inadequacy  in  training  or  T.O.s  will  be  reported. 

Qualified  test  team  members  will  verify  the  deficiency,  determine  its  impact,  and  document 
the  findings  in  the  QOT&E  Records.  Test  team  members  will  justify  any  discrepancy 
between  AFSC  job  description  and  tasks  performed.  Data  collected  during  the 
maintainability  demonstration  may  be  used  to  supplement  test  results. 

3.7.4.  Method  of  Evaluation.  All  logistics  supportability,  readiness,  maintainability, 
training,  and  technical  data  will  be  extracted,  summarized,  and  compared  to  the 
performance  criteria.  Course  instructors  will  assess  the  adequacy  of  the  course  chart,  plan 
of  instruction,  provided  student  training  material,  and  training  program  and  training 
equipment  plan.  Near  the  end  of  the  QOT&E,  all  analysts,  programmers,  and  technicians 
will  rate  the  training  and  T.O.s  as  either  "adequate"  or  "not  adequate."  Any  "not  adequate" 
rating  will  be  fully  justified.  MOP  7-27  was  determined  to  be  critical  to  mission 
accomplishment  and  must  meet  user  criteria  to  rate  the  COI  as  "resolved  satisfactory."  In 
addition,  MOPs  marked  as  "significant"  were  determined  to  have  a  significant  impact  on 
mission  accomplishment.  To  cause  the  COI  to  be  rated  "resolved  unsatisfactory,"  the 
results  of  these  MOPs  must  fail  to  meet  user  criteria  and  prevent  mission  accomplishment. 
Any  purposely  induced  failures  will  not  be  included  when  figuring  MTBCF,  MTBMc,  and 
MDT.  The  following  formulas  will  be  used: 

MTBCF  =  NUMBER  OF  OPERATING  HOURS 
NUMBER  OF  CRITICAL  FAILURES 

NOTE:  a  critical  failure  is  any  significant  outage  as  defined  in 
AFSFCR  105-1,  Attachment  4  (refer  to  Figure  3-1). 

MTBMc  =  NUMBER  OF  OPERATING  HOURS 

NUMBER  OF  CORRECTIVE  MAINTENANCE  ACTIONS 

NOTE:  corrective  maintenance  is  only  unscheduled  maintenance  events. 

MRT  =  TOTAL  TIME  TO  REPAIR  ALL  CORRECTIVE  MAINTENANCE  EVENTS 
NUMBER  OF  CORRECTIVE  MAINTENANCE  EVENTS 

NOTE:  Repair  time  starts  when  the  technician  arrives  and  continues  until  the  repair  is 

completed,  or  a  part  is  determined  to  be  required.  It  does  not  include  maintenance 


delays  (ordering  parts,  technician  travel  time,  etc.)  or  supply  delays. 

Maximum  TTR  =  MAXIMUM  REPAIR  TIME  AS  DEFINED  IN  MRT  ABOVE 


TABLE  3-1.  REQUIRED  MESSAGES  AND  HEADERS 


1. 

SOON  MANOP  HEADERS 

HEADER 

TYPE  MESSAGE 

HOHW  PHFF  DDGGgg 

Scheduled  PLAIN  message  (file  time) 

HOHW1  PHFF  DDGGgg 

Preliminary  message  for  FLARE,  DALAS,  PLAIN, 
HSTRY,  and  STATS  code  types. 

HOHW2  PHFF  DDGGgg 

End  of  day  summary  and  final  STATS  message. 

HOHW3  PHFF  DDGGgg 

SPOTS  message 

HOHW5  PHFF  DDGGgg 

Equipment  status  summary  message  (PLAIN) 

HOHW6  PHFF  DDGGgg 

BXOUT  message 

2. 

RSTN  MANOP  HEADERS 

HEADER 

TYPE  MESSAGE 

HRHW1  PHFF  DDGGgg 

Preliminary  messages  for  BURST,  SWEEP,  STATS, 
and  PLAINS. 

HRHW2  PHFF  DDGGgg 

End  of  day  summary  and  final  STATS  message. 

HRHW3  PHFF  DDGGgg 

IFLUX  message 

3.  EVENT  MESSAGE  MANOP  HEADERS 

HEADER 

TYPE  MESSAGE 

HEHW  PHFF  DDGGgg 
(5  bells) 

Ail-sensor  message 

HEHW1  PHFF  DDGGgg 
(5  bells) 

Preliminary  EVENT  message  for  FLARE,  DALAS, 
BURST.  SWEEP,  and  SITEC 

TABLE  3-1.  REQUIRED  MESSAGES  AND  HEADERS 
(CONTINUED) 


4.  IONOSONDE  MESSAGES 

HEADER 

TYPE  MESSAGE 

HIHW3  PHFF  DDGGgg 

TELCO  messages 

HIHW4  PHFF  DDGGgg 

IONSS  messages 

5.  SERVICE  MESSAGES 

HEADER 

TYPE  MESSAGE 

Not  Specific 

Not  Specific 

3.8.  COl-8:  Does  the  SEON  facility  provide  adequate  power  and  environmental  support? 

3.8.1.  Scope.  The  uninterruptable  power  supply  (UPS),  environmental  control  system, 
effects  of  electro-magnetic  interference,  and  effects  of  radio  frequency  interference  will  be 
evaluated  during  normal  operation  and  maintenance  each  day  during  the  90-day  QOT&E. 
MOPs  8-1  and  8-2  are  the  critical  MOPs  supporting  this  COl. 

3.8.2.  MOPs  and  Performance  Criteria: 

3.8.2. 1.  MOP  8-1:  time  the  UPS  provides  power  to  the  equipment  connected  to  it  in  a 
generator  backed-up  configuration.  (CRITICAL) 

Performance  Criteria:  long  enough  for  the  backup  generator  to  come  on-line  with  no  loss 
of  SEON  data. 

3.8. 2. 2.  MOP  8-2:  time  the  UPS  provides  power  to  the  equipment  connected  to  it  in  a 
non-generator  backup  configuration.  (CRITICAL) 

Performance  Criterion:  long  enough  for  the  equipment  to  be  orderly  and  safely 
powered-down  with  no  loss  of  SEON  data. 

3.8. 2.3.  MOP  8-3:  range  of  temperature  in  the  SOON  equipment  room.  (SIGNIFICANT) 
Performance  Criterion:  60"  F  to  80"  F. 

3.8. 2.4.  MOP  8-4:  range  of  temperature  in  the  RSTN  equipment  room.  (SIGNIFICANT) 
Performance  Criterion:  50"  F  to  90"  F. 


3.X. 2.5.  MOP  X-5:  range  of  relative  humidity  in  the  SOON  equipment  room. 
(SIGNIFICANT) 

Performance  Criterion:  30 9r  to  60 c/< . 

3.X.2.6.  MOP  X-6:  range  of  relative  humidity  in  the  RSTN  equipment  room. 
(SIGNIFICANT) 

Performance  Criterion:  25‘/r  to  75 7c. 

3.X.2.7.  MOP  X-7:  The  amount  of  operational  impact  caused  by  EMI  encountered  by  the 
SEON.  (SIGNIFICANT) 

Performance  Criterion:  No  operational  impact. 

3.8.2.X.  MOP  X-X:  The  amount  of  operational  impact  caused  by  RF1  encountered  by  the 
SEON.  (SIGNIFICANT) 

Performance  Criterion:  No  operational  impact. 

3.X.3.  Mission  Scenarios.  The  Test  Director  will  coordinate  all  power  outages  with  the 
Palehua  Site  Commander.  To  test  the  UPS  capability,  power  outages  will  be  induced  with 
the  systems  configured  with  and  without  generator  backup.  Once  power  outages  occur,  the 
Test  Director  will  time  how  long  it  takes  for  site  personnel  to  properly  shutdown  the 
system,  or  the  standby  generator  to  come  on-line.  Existing  hydrothermographs  in  the 
RSTN  and  SOON  equipment  rooms  will  be  used  to  document  the  temperature  and  relative 
humidity.  The  recorded  readings  on  the  hydrothermographs  will  be  examined  daily.  If  an 
out-of-toierance  condition  is  found,  it  will  be  clearly  marked  on  the  chart.  All  relative 
information  will  be  documented  in  the  Test  Director's  QOT&E  Record. 

3.X.4.  Method  of  Evaluation.  All  environmental,  power,  EMI,  and  RF1  data  will  be 
extracted,  summarized,  and  compared  to  the  performance  criteria.  All  critical  MOPs  must 
meet  user  requirements  in  order  to  rate  this  COI  as  "resolved  satisfactory."  In  addition, 
MOPs  marked  as  "significant"  were  determined  to  have  a  significant  impact  on  mission 
accomplishment.  To  cause  the  COI  to  be  rated  "resolved  unsatisfactory,"  the  results  of 
these  MOPs  must  fail  to  meet  user  criteria  and  prevent  mission  accomplishment. 

3.9.  COI-9:  Is  the  human  factors  engineering  of  the  SEON  adequate? 

3.9.1.  Scope.  The  Human  Factors  Engineering  (HFE)  will  be  evaluated  during  the 
entire  90-day  QOT&E.  All  SEON  aspects  applicable  to  the  Military  Standard  1472D, 
Human  Engineering  Design  Criteria  for  Military  Systems,  Equipment,  and  Facilities  will  be 
evaluated.  MOP  9-1  is  the  critical  MOP  for  this  COI. 


3.9.2.  MOPs  and  Performance  Criteria: 


3.9.2. 1 .  MOP  9-1:  number  of  significant  outages  caused  by  improper  HFE  considerations. 
(CRITICAL) 

Performance  Criterion:  zero  detected. 

3.9. 2.3.  MOP  9-2:  subjective  assessment  by  qualified  operators,  maintainers,  and  analysts 
of  the  HFE  characteristics.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

3.9.3.  Mission  Scenarios.  All  significant  outages  as  defined  in  Air  Force  Space  Forecast 
Center  Regulation  (AFSFCR)  105-1,  Attachment  4  (refer  to  Figure  3-1)  system  will  be 
documented  using  established  procedures  and  in  the  QOT&E  Operations  and  Maintenance 
Records.  Near  the  end  of  the  QOT&E,  the  analysts',  programmers,  and  technicians  will 
complete  an  HFE  Questionnaire. 

3.9.4.  Method  of  Evaluation.  At  the  end  of  each  section  of  the  HFE  Questionnaire,  the 
test  team  will  rate  the  area  as  either  "adequate”  or  "not  adequate"  based  on  their 
professional  judgement.  All  data  relating  to  HFE  will  be  summarized  and  compared  to  the 
performance  criteria.  MOP  9-1  must  meet  user  requirements  in  order  to  rate  this  COI  as 
"resolved  satisfactory."  In  addition,  MOP  9-2,  marked  as  "significant"  was  not  deemed 
"critical,"  but  was  determined  to  have  a  significant  impact  on  mission  accomplishment.  To 
cause  the  COI  to  be  rated  "resolved  unsatisfactory,"  the  results  of  this  MOP  must  fail  to 
meet  user  criteria  and  prevent  mission  accomplishment. 

3.10  Survivability  Assessment.  The  users  have  determined  that  survivability  of  the  system 
isn't  an  issue.  The  SEON  isn't  expected  nor  designed  to  operate  in  a  hostile  environment. 

It  isn't  hardened  to  operate  in  a  chemical,  biological,  or  radiological  environment. 


SECTION  4  -  ADMINISTRATION 


4.0.  TEST  MANAGEMENT.  Headquarters  (HQ)  Air  Force  Materiel  Command  (AFMC) 
is  the  implementing  command.  AFMC  has  tasked  Sacramento  Air  Logistics  Center 
(SM-ALC)  with  System  Program  Office  (SPO),  and  supporting  command  responsibilities. 
The  participating  commands  directly  involved  in  the  Qualification  Operational  Test  and 
Evaluation  (QOT&E)  are:  HQ  Air  Weather  Service  (AWS),  Air  Force  Operational  Test 
and  Evaluation  Center  (AFOTEC),  and  HQ  Air  Force  Space  Command  (AFSPACECOM). 
The  operating  commands  directly  supporting  the  QOT&E  are:  HQ  AWS,  HQ  AFSPACE¬ 
COM,  HQ  Pacific  Air  Forces  (PACAF),  HQ  Air  Combat  Command  (ACC),  and  HQ  United 
States  Air  Forces  Europe  (USAFE). 

4.1.  TASKING. 

4.1.1.  Detachment  (Det)  1,  AFOTEC  has  assigned  a  Test  Director.  The  Test  Director  has 
overall  responsibility  for  planning,  conducting,  and  reporting  the  QOT&E.  He  will 
schedule  required  resources  through  the  test  resource  plan  (TRP).  He  will  have  operational 
control  over  the  test  team  during  the  QOT&E.  He  will  convene  the  test  data  scoring  board 
(TDSB)  at  the  test  site  to  validate  the  reliability  and  maintainability  data  collected  during 
the  QOT&E.  Assisting  him  from  Det  1,  AFOTEC  is  the  Associate  Test  Director(s), 
Software  Test  Manager  (STM),  Associate  STM,  and  Statistician. 

4. 1. 1.1.  The  Associate  Test  Director  will  assist  the  Test  Director.  In  the  absence  of  the 
Test  Director,  he  will  assume  responsibility  for  planning,  conducting,  and  reporting  the 
QOT&E. 

4. LI. 2.  The  STM  will  plan  and  conduct  the  software  evaluations  consisting  of 
Maintainability,  Usability,  Support  Resources,  Maturity,  and  Life  Cycle  Process.  He  will 
ensure  an  adequate  number  of  software  evaluators  are  used  for  the  Software  Maintainability 
Evaluation.  The  STM  will  provide  all  plans  and  reports  to  the  Test  Director  as  soon  as 
possible,  but  no  later  than  30  days  after  last  test  event. 

4. 1.1.3.  The  Associate  STM  will  assist  the  STM  in  conducting  the  various  software 
evaluations.  In  the  absence  of  the  STM,  he  will  assume  responsibility  for  the  software 
evaluations. 

4. 1.1. 4.  The  Statistician  will  ensure  the  accuracy  and  meaningfulness  of  all  statistical  data. 

4.1.2.  SM-ALC/LHFBB  is  the  SPO.  The  SPO  will  prepare  and  coordinate  the  Test  and 
Evaluation  Master  Plan  (TEMP).  The  program  manager  certifies  the  upgraded  Solar 
Electro-Optical  Network  (SEON)  at  the  Palehua  Solar  Observatory  as  ready  for  dedicated 
QOT&E  prior  to  test  start.  The  program  manager  will  ensure  all  actions  required  in 
paragraph  4-7  of  Technical  Order  00-35D-54  are  taken.  Assisting  with  the  QOT&E  from 
SM-ALC  will  be  a  system  engineer  and  software  consultant. 


4. 1.2.1.  The  System  Engineer  will  assist  the  Test  Director  in  assuring  the  QOT&E  plan 
uses  acceptable  system  parameters  anil  addresses  all  operational  effectiveness  and  suitability 
issues.  He  will  also  assist  the  SPO  in  determining  actions  required  to  correct  deficiencies 
noted  during  the  QOT&E. 

4. 1.2.2.  The  Software  Consultant  will  work  with  the  STM  to  determine  that  all  software 
issues  are  adequately  addressed  in  the  QOT&E  plan.  He  will  also  participate  in  the 
Software  Maintainability  Evaluation. 

4.1.3.  HQ  AWS/PMT  has  assigned  an  acquisition  manager.  The  acquisition  manager  will 
be  the  focal  point  for  ail  operational  :ssues  regarding  the  SEON.  He  will  ensure  the 
QOT&E  plan  adequately  addresses  all  operational  effectiveness  and  suitability  issues.  He 
will  ensure  the  required  TRP  identified  test  team  members  are  provided  to  support  the 
QOT&E.  Subordinate  to  HQ  AWS  is  the  Air  Force  Space  Forecast  Center  (AFSFC);  Det 
5,  AFSFC  at  the  Palehua  Solar  Observatory:  and  the  operating  units  assigned  to  the  other 
SEON  sites. 

4. 1.3.1.  The  AFSFC  will  help  HQ  AWS  ensure  the  QOT&E  plan  adequately  addresses  all 
operational  effectiveness  and  suitability  issues.  They  will  ensure  sufficient,  validated  data 
from  other  SEON  sites  and  Culgoora  Solar  Observatory  in  Australia  is  provided  to  the 
Palehua  Solar  Observatory  to  verify  the  data  collected  by  the  upgraded  SEON.  They  will 
provide  a  representative  to  sit  on  the  TDSB. 

4. 1.3.2.  Det  5,  AFSFC  at  the  Palehua  Solar  Observatory  will  assist  the  AFSFC  in  ensuring 
the  QOT&E  plan  adequately  addresses  all  operational  effectiveness  and  suitability  issues. 
They  will  provide  qualified  programmers  and  analysts  as  indicated  in  the  TRP  to  support 
the  QOT&E.  They  will  also  provide  all  resources  agreed  to  in  the  letter  of  agreement. 

They  will  ensure  the  upgraded  SEON  has  a  comprehensive  safety  inspection  after 
installation  and  prior  to  the  start  of  QOT&E.  They  will  provide  a  representative  to  sit  on 
the  TDSB. 

4.1.4.  HQ  AFSPACECOM/LKLMK  is  the  SEON  maintenance  focal  point.  They  will 
ensure  the  QOT&E  plan  adequately  addresses  all  operational  effectiveness  and  suitability 
issues.  They  will  provide  the  test  team  members  as  indicated  in  the  TRP,  including  all 
technicians  required  for  the  maintainability  demonstration.  They  will  also  ensure  training 
slots  are  allocated  for  all  test  team  members,  including  the  Test  Director,  Associate  Test 
Director,  STM,  and  Associate  STM.  They  will  provide  a  representative  to  sit  on  the 
TDSB. 

4.1.5.  The  49th  Communications  Squadron  (49  CS/LGR)  is  the  SOON  centralized  repair 
activity  (CRA).  The  1957th  Communications  Group  (15  CS/LGR)  is  the  Radio  Solar 
Telescope  Network  (RSTN)  CRA.  They  will  ensure  the  QOT&E  plan  adequately  addresses 
all  operational  effectiveness  and  suitability  issues.  They  will  provide  a  representative  to  sit 
on  the  TDSB. 


4.2.  TRAINING  REQUIREMENTS.  Test  team  members  performing  programming, 
operations,  and  maintenance  will  be  qualified  on  the  current  SEON.  The  contractor  will 
provide  training  on  operations,  programming,  and  maintenance  of  the  upgraded  SEON  prior 
to  QOT&E.  HQ  AFSPAC'EGOM/LKLMK  is  the  point  of  contact  (POC)  for  the  training. 
The  Test  Director  will  provide  a  1  -day  QOT&E  training  class  immediately  prior  to  the 
QOT&E  to  teach  the  test  team  members  data  collection  processes  anil  the  goals  of  the 
QOT&E. 

4.3.  SAFETY. 

4.3.1.  Safety  will  not  be  compromised  to  accomplish  test  objectives.  The  Test  Director  is 
responsible  for  the  overall  QOT&E  safety  program.  All  personnel  will  receive  a 
comprehensive  safety  briefing  before  the  start  of  the  QOT&E.  The  Hickam  Air  Force 
Base,  Hawaii  Safety  Office  will  be  consulted  for  support  in  evaluating  and  correcting  any 
unsafe  condition.  They  will  also  be  consulted  for  any  required  incident  report.  If  an 
unsafe  or  potentially  hazardous  condition  arises  during  the  QOT&E.  all  testing  will  be 
suspended  until  the  unsafe  condition  is  corrected  or  controlled. 

4.3.2.  Before  commencing  the  QOT&E,  Palehua  Solar  Observatory  Safety  Officer  w  ill 
ensure  the  Hickam  AFB,  Hawaii  Safety  Office  conducts  a  comprehensive  safety  review . 

The  Test  Director  will  ensure  all  reviews  become  part  of  the  project  file.  If  the  test 
objectives  or  methods  are  added  or  altered,  a  safety  review  will  be  conducted.  All  test 
team  members  must  comply  with  applicable  military  department  (MiLDEP).  major 
command,  and  test  location  safety  directives  and  procedures. 

4.3.3.  If  a  safety-related  incident  occurs,  the  Test  Director,  test  team  members,  and  Palehua 
Solar  Observatory  Safety  Officer  will  ensure  the  reports  required  by  Air  Force  Regulation 
(AFR  i  127-4  are  completed.  The  Test  Director  wall  verify  the  hazard  severity  category 
with  the  Palehua  Solar  Observatory  Safety  Officer  prior  to  releasing  any  service  report. 

The  QOT&E  report  will  contain  the  results,  conclusions,  and  recommendations  of  the  safety 
objective.  This  evaluation  will  be  based  upon  observation  of  safety  procedures,  hazard 
identification  forms,  and  incident/accident  reports. 

4.4.  SECURITY.  Paragraph  1.6  of  the  Program  Management  Plan  (PMP),  dated 
30  Mar  X9  states:  "Ail  efforts  currently  directed  under  this  program  are  unclassified  for 
official  use  only.  This  includes  equipment,  details  of  equipment  performance,  and 
specifications.  The  Program  Office  has  determined  that  a  Security  Classification  Guide  for 
SEON  is  not  required."  Communications  Security  (COMSEC)  support  and  control  of 
compromising  emissions  (TEMPEST)  protection  aren't  required.  The  physical  security 
requirements  already  in  place  at  the  Palehua  Solar  Observatory  are  adequate  for  this 
QOT&E.  Personnel  desiring  to  visit  the  test  location  during  the  QOT&E  for  purposes 
directly  related  to  the  test  effort  must  coordinate  the  visit  through  the  Test  Director. 
Personnel  visiting  the  test  site  must  not  impede  the  progress  of  the  QOT&E.  Phis  program 
has  been  determined  to  be  insusceptible  to  hostile  intelligence  exploitation;  however,  the 

4  t 


test  director  will  ensure  sound  security  practices  are  implemented  during  the  QOT&E  and 
wili  continue  to  monitor  the  program  for  changes  that  could  reveal  system  capabilities  or 
vulnerabilities  to  hostile  intelligence  sources. 

4.5.  ENVIRONMENTAL  IMPACT.  The  operational  test  and  evaluation  will  be  conducted 
in  conjunction  with  the  SEON  upgrade  at  Palehua  Solar  Observatory.  The  environmental 
impact  is  covered  by  the  IX42d  Engineering  Installation  Group  Revised  Project  Support 
Agreement,  Attachment  I,  paragraph  4.d..  dated  10  Feb  92.  It  states  "environmental 
impact  considerations,  none." 

4.6.  RELEASE  OF  INFORMATION.  Prior  to  publishing  the  results  of  the  QOT&E,  all 
information  will  be  treated  as  "FOR  OFFICIAL  USE  ONLY."  The  Test  Director  will  brief 
all  test  team  members  not  to  discuss  any  preliminary  QOT&E  results  prior  to  the  release  of 
the  QOT&E  report.  The  QOT&E  report  will  have  unlimited  distribution. 

4.7.  FOREIGN  DISCLOSURE.  Any  release  of  test  and  evaluation  information  to  foreign 
nationals,  governments,  or  agencies  must  be  approved  by  HQ  USAF/CVAII  through 

HQ  AFOTEC/SP. 


4-4 


SECTION  5  -  REPORTING 


5.0.  ACTIVITY  REPORTS.  The  Test  Director  will  send  activity  icports  as  required. 

5.1.  STATUS  REPORTS.  The  Test  Director  will  send  a  status  report  as  required.  These  reports 
will  be  sent  bi-weekly  as  a  minimum  during  the  Qualification  Operational  Test  and  Evaluation 
(QOT&E).  Status  reports  will  be  addressed  as  follows: 

FROM:  DET  1  AFOTEC  SCOTT  AFB  IL//TA//*** 

TO:  SM  AXiC  MCCLELLAN  AFB  CA//LHFBB// 

INFO:  HQ  AFOTEC  KIRTLAND  AFB  NM//TK// 

HQ  AFSPACECOM  PETERSON  AFB  CO//LKLMK// 

AFSFC  FALCON  AFB  CO/ /DOO/ / 

HQ  AWS  SCOTT  AFB  IL//PMT// 

DET  1  AFOTEC  SCOTT  AFB  IL//TAA// 

4  9CS  HOLLOMAN  AFB  NM//SCO// 

DET  5  AFSFC  HICKAM  AFB  HI//CC/DO// 

15CS  HICKAM  AFB  HI / / SCL/ SCLR/ / 

5.2.  SIGNIFICANT  EVENT  REPORT.  The  Test  Director  will  send  a  significant  event 
report  as  required. 

5.3.  FINAL  QUALIFICATION  OPERATIONAL  TEST  AND  EVALUATION  (QOT&E) 
REPORT.  The  Test  Director  will  prepare  this  report  with  the  assistance  of  the  test  team. 

AH  data  collected  will  be  analyzed  before  presenting  the  results,  conclusions,  and 
recommendations.  The  QOT&E  report  will  be  approved  no  later  than  60  calendar  days 
following  the  last  test  event. 

5.4.  BRIEFINGS.  If  requested,  briefings  will  be  conducted  at  the  conclusion  of  the 
QOT&E.  Briefing  locations  will  be  coordinated  through  the  Test  Director. 

5.5.  PRODUCT  QUALITY  DEFICIENCY  REPORTS  (PQDRs). 

5.5.1.  PQDRs  will  be  submitted  in  accordance  with  Technical  Order  00-35D-54.  They 
will  be  sent  to  the  appropriate  agency  and  included  in  the  final  test  report.  PQDRs  will  be 
addressed  as  follows: 

FROM:  DET  1  AFOTEC  SCOTT  AFB  IL//TA//*** 

TO:  SM  ALC  MCCLELLAN  AFB  CA/ /SE/LHFBB/ / 

INFO:  HQ  AFOTEC  KIRTLAND  AFB  NM//SE/TK//* 

AFISC  NORTON  AFB  CA//DR//** 

HQ  USAF  WASHINGTON  DC//SE//* 

HQ  AFSPACECOM  PETERSON  AFB  CO/ /LKLMK/SE/ / * 

AFSFC  FALCON  AFB  CO //DOO// 

HQ  AWS  SCOTT  AFB  IL/ / PMT/ SE/ / * 

4  9CS  HOLLOMAN  AFB  NM//SCO// 
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DET  1  AFOTEC  SCOTT  AFB  IL//TAA// 

ZEN  X5CS  HICKAM  AFB  HI//SCLR/SE//* 

NOTES: 

*  Safety  offices  (SE)  identified  with  an  asterisk  (*)  will  receive  only  Category  I  and 
Category  II  Hazard  PQDRs. 

**  AFISC  will  only  receive  Category  1  and  Category  11  Hazard  PQDRs. 

***  Since  the  test  team  will  be  deployed  to  the  Palehua  Solar  Observatory  during  the  test, 
the  "FROM"  address  may  be  changed. 

5.5.2.  Report  Control  Number  (RCN).  The  PQDR  screening  point  (test  team)  will  validate 
and  assign  a  unique  RCN  to  each  PQDR  generated  during  the  QOT&E.  Each  RCN  will  be 
structured  as  follows:  FA3103  YY  XXXX  A  SEON,  where: 

FA3103  =  Unique  six  character  alphanumeric  DOD  Activity  Address  Code  for 
Detachment  1,  Air  Force  Operational  Test  and  Evaluation  Center 
(Det  1,  AFOTEC). 

XX  =  Last  two  digits  of  the  calendar  year. 

YYYY  =  Chronological  order  of  occurrence  (0001  through  9999). 

A  =  Identifies  the  test  agency  branch  (Det  1,  AFOTEC/TAA). 

SEON  =  Program  identifier. 

5.5.3.  PQDR  Prioritization.  PQDR  prioritization  will  start  at  the  beginning  of  the  QOT&E. 
Deficiency  and  Enhancement  Analysis  Ranking  Technique  (DART)  will  be  used  to 
prioritize  the  PQDRs.  Before  the  start  of  the  QOT&E,  a  DART  board  will  be  established 
consisting  of  the  following  members: 

5.5.3. 1.  The  SEON  QOT&E  Test  Director. 

5. 5. 3. 2.  The  SEON  QOT&E  Associate  Test  Director(s). 

5. 5. 3. 3.  A  representative  from  the  Centralized  Repair  Activity  (CRA). 

5. 5. 3.4.  A  representative  from  Det  5,  Air  Force  Space  Forecast  Center. 

5. 5. 3. 5.  A  representative  from  Air  Force  Space  Command/LKLMK,  SEON  maintenance 
focal  point. 

5. 5. 3. 6.  A  representative  from  Air  Weather  Service  program  office  (AWS/PMT). 
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5.5.4.  Material  Improvement  Projects  (MIP)  Tracking.  In  accordance  with  TO  OU-35D-54. 
paragraph  4-1  1.1,  the  test  team  will  track  all  MiPs  generated  due  to  PQDRs.  The  Test 
Director  will  assign  a  MIP  focal  point  who  will  track  the  status  of  open  MIPs  using  the 
MiP  data  base  developed  by  the  test  team. 
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COI  and  MOE/MOP  Matrix 


t.  Measures  of  Effectiveness  (MOEs)  have  been  created  for  the  SEON  I  QOT&E  based  on 
changes  to  AFOTECR  55-9,  by  Policy  Letter  93-01. 

2.  The  following  is  the  new  COI  and  MOE/MOP  matrix: 

COM:  Is  the  upgraded  SEON  safe  to  operate  and  maintain? 

MOE  1-1:  Adequacy  of  the  system  design  to  allow  safe  operations  and  maintenance. 

MOP  1-1-1:  number  of  SEON  safety  hazards  that  meet  Category  (CAT)  I  deficiency  PQDR 
criteria.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  1-1-2:  number  of  SEON  safety  hazards  that  meet  CAT  II  deficiency  PQDR  criteria. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

COI-2:  Do  the  SEON  computers  provide  sufficient  capabilities  to  support  the  mission? 

MOE  2-1:  Capability  of  the  SEON  computers. 

MOP  2-1-1:  number  of  real-time  computer  processors  that  cannot  be  restored  using  the 
Ethernet.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  2-1-2:  number  of  workstations  that  cannot  be  restored  to  full  mission  capability  from 
other  nodes  over  the  ethernet.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  2-1-3:  number  of  computers  that  cannot  be  restored  to  full  mission  capability  using  the 
tape  backups.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  2-1-4:  number  of  interfaces  not  achieved  between  the  computers  and  equipment. 
(CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  2-1-5:  number  of  menu  items  that  cannot  be  selected  by  a  single  keystroke  or  a  single 
combination  of  two  keystrokes  (as  in  alt/F10).  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 


MOP  2-1-6:  number  of  mission  data  segment  changes  not  executable  via  operator  menu 
selection. 

Performance  Criterion:  zero  detected. 

MOP  2-1-7:  number  of  equipment  checks  and  alignments  not  executable  via  keyboard  entry. 
Performance  Criterion:  zero  detected. 

MOP  2-1-X:  percent  of  selected  text  files  adequately  printed  to  the  LN05R  printers. 
(SIGNIFICANT) 

Performance  Criterion:  97%. 

MOP  2-1-9:  percent  of  selected  graphics  files  adequately  printed  to  the  LN05R 
printers.  (SIGNIFICANT) 

Performance  Criterion:  97%. 

MOP  2-1-10:  percent  of  screen  images  adequately  printed  to  the  LN05R  printers. 
(SIGNIFICANT) 

Performance  Criterion:  97%. 

MOP  2-1-11:  number  of  inadequate  plots  from  the  SFIR.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  2-1-12:  number  of  Digital  Command  Language  (DCL)  level  functions  that  cannot  be 
performed  by  the  operators. 

Performance  Criterion:  zero  detected. 

MOP  2-1-13:  number  of  nodes  that  cannot  be  assigned  or  reassigned  without  bringing  the 
system  down.  (CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  2- 1- 14:  time  required  for  operators  to  load  and  run  new  version  software. 
(CRITICAL) 

Performance  Criterion:  less  than  X  hours. 

MOE  2-2:  Capability  of  the  computer  to  interface  with  LIAs. 

MOP  2-2-1:  performs  required  functions  where  the  computer  individually  reads,  writes, 
interacts,  and  controls  for  each  lock  in  amplifier  (LIA).  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOE  2-3:  Capability  of  the  system’s  built-in  tests. 

MOP  2-3-1:  percent  of  faults  detected  by  the  automatic  monitoring  system. 

(SIGNIFICANT )  Performance  Criterion:  a  minimum  of  99%. 


MOP  2-3-2:  false  alarm  rate  of  faults  by  the  automatic  monitoring  system.  (SIGNIFICANT) 
Performance  Criterion:  a  maximum  of  I  %. 

MOP  2-3-3:  number  of  incorrect  status  indications.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOE  2-4:  Capability  of  the  interface  between  the  SOON  and  DIPS. 

MOP  2-4- 1 :  number  of  workstations  assigned  as  system  console  that  cannot  perform 
calibration  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

MOP  2-4-2:  number  of  workstations  assigned  as  system  console  that  cannot  control  the  data 
image  collection  function  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

MOP  2-4-3:  number  of  workstations  assigned  as  system  console  that  cannot  control  the 
scheduling  of  movie  replay  function  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

MOP  2-4-4:  number  of  workstations  assigned  as  system  console  that  cannot  control  the 
remote  shutdown  of  the  DIPS. 

Performance  Criterion:  zero  detected. 

MOP  2-4-5:  number  of  inadequate  magnetic  contour  and  Stoneyhurst  plots  performed  by  the 
system  using  data  provisioned  from  the  telescopes  under  control  of  the  workstations. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

COI-3:  Does  the  upgraded  RSTN  allow  detection  of  solar  events? 

MOE  3-1:  Capability  of  the  RSTN  swept  frequency  interferometric  radiometer  (SFIR) 
subsystem. 

MOP  3-1-1:  percent  of  solar  bursts  detected  by  the  current  SFIR  between  30-75  megahertz 
(MFIz)  that  are  also  detected  by  the  new  SFIR  between  30-75  MHz,  while  both  SFIRs  are 
operational.  (SIGNIFICANT) 

Performance  Criterion:  97%. 

MOP  3-1-2:  percent  of  solar  bursts  detected  by  other  SFIRs  (at  other  SEON  sites)  between 
30-75  MHz  that  are  also  detected  by  the  new  SFIR.  (SIGNIFICANT) 

Performance  Criterion:  95%  of  bursts  detected  by  other  sites  during  concurrent  patrol 
periods. 


MOP  3-1-3:  subjective  rating  by  qualified  analysts  of  solar  sweep  signatures  detected 
between  30-75  MHz.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  3-1-4:  subjective  rating  by  qualified  analysts  of  solar  sweep  intensities  between 
30-75  MHz.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  3-1-5:  speed  of  Type  2  shocks.  (SIGNIFICANT) 

Performance  Criterion:  within  500  kilometers  per  second  of  the  speed  indicated  by  the 
current  system. 

MOP  3-1-6:  minimum  sensitivity  of  the  new  SFIR  at  30  MHz.  (SIGNIFICANT) 
Performance  Criterion:  10  Solar  Flux  Units  (SFUs). 

MOP  3-1-7:  minimum  sensitivity  of  the  new  SFIR  at  250  MHz.  (SIGNIFICANT) 
Performance  Criterion:  40  SFUs. 

MOP  3-1-X:  number  of  solar  events,  types  of  solar  burst  signatures,  and  burst  intensities 
between  75-250  MHz  detected  by  the  SFIR.  There  is  no  performance  criterion  for  this  MOP. 

MOE  3-2:  Capability  of  the  RSTN's  discrete  frequency  radiometer. 

MOP  3-2-1:  percent  of  solar  intensity  difference  between  that  indicated  by  the  discrete 
frequency  radiometer  computer  and  the  charted  intensity.  (SIGNIFICANT) 

Performance  Criterion:  10%  or  less. 

MOP  3-2-2:  the  computer  reported  burst  characteristics  (start  time,  maximum  time,  stop 
time,  mean  flux,  integrated  flux,  and  peak  flux)  compared  to  the  charted  characteristics. 
(SIGNIFICANT) 

Performance  Criteria:  plus  or  minus  5  seconds  for  times  and  within  10%  for  mean  flux, 
integrated  flux,  and  peak  flux. 

MOP  3-2-3:  number  of  solar  bursts  the  discrete  frequency  radiometer  incorrectly  identifies 
(excluding  RFI).  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  3-2-4:  subjective  rating  by  experienced  RSTN  analysts  and  programmers  of  discrete 
frequency  bursts.  (CRITICAL) 

Performance  Criterion:  adequate. 

COI-4:  Does  the  upgraded  SOON  allow  detection  of  solar  events? 

MOE  4-1:  Capability  of  the  SOON  charged  couple  devices  (cameras)  and  their  interfaces. 
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MOP  4-1-1:  subjective  rating  by  qualified  analysts  of  H-Alpha  images.  (SIGNIFICANT) 
Performance  Criterion:  adequate. 

MOP  4-1-2:  minimum  resolution  of  large-scale  H- Alpha  images. 

(SIGNIFICANT) 

Performance  Criterion:  2.2  arc  seconds. 

MOP  4-1-3:  minimum  resolution  of  full  disk  H- Alpha  images. 

(SIGNIFICANT) 

Performance  Criterion:  12.5  arc  seconds. 

MOP  4-1-4:  subjective  rating  by  qualified  analysts  of  spectrographic  images.  (CRITICAL) 
Performance  Criterion:  adequate. 

MOP  4-1-5:  minimum  resolution  of  large-scale  spectrographic  images.  (SIGNIFICANT) 
Performance  Criterion:  2.2  arc  seconds. 

MOP  4-1-6:  minimum  resolution  of  full  disk  spectrographic  images.  (SIGNIFICANT) 
Performance  Criterion:  12.5  arc  seconds. 

MOP  4-1-7:  subjective  rating  by  qualified  analysts  of  H- Alpha  images  transferred  to  the 
DIPS  from  the  RSI 70  output  of  the  CCD.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  4-1-8:  subjective  rating  by  experienced  SOON  analysts  and  programmers  of  solar  flare 
analysis  function.  (CRITICAL) 

Performance  Criterion:  adequate. 

MOE  4-2:  Capabilities  of  the  digital  imagery  processing  subsystem  (DIPS). 

MOP  4-2-1:  subjective  rating  by  qualified  analysts  of  spectrographic  magnetic  data 
transferred  to  the  DIPS. 

Performance  Criterion:  adequate. 

MOP  4-2-2:  number  of  frames  transferred  to  the  DIPS  from  the  SOON.  (SIGNIFICANT) 
Performance  Criterion:  a  minimum  of  12  frames  per  minute. 

MOP  4-2-3:  DIPS  playback  speed. 

Performance  Criteria:  up  to  30  frames  per  second. 

MOP  4-2-4:  number  of  frames  that  can  be  stored  by  the  DIPS. 

Performance  Criterion:  a  minimum  of  800  frames. 
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MOP  4-2-5:  resolution  of  each  frame  (stored  in  the  DIPS). 

Performance  Criterion:  a  minimum  of  512  X  512  X  X  bits  per  frame. 

MOP  4-2-6:  sufficiency  of  shades  of  gray  (of  stored  DIPS  images). 

Performance  Criterion:  256  gray  shades  to  allow  the  capability  for  coronal  hole  detection  at 
10X30  angstroms. 

MOP  4-2-7:  speed  of  overlaying  pictures  on  stored  grids  (in  the  DIPS). 

Performance  Criterion:  a  minimum  of  one  frame  per  second. 

MOP  4-2-X:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the  histogram 
capability.  (CRITICAL) 

Performance  Criterion:  adequate. 

MOP  4-2-9:  minimum  resolution  of  H-Alpha  and  visible  light  (including  Mg  B;). 
(SIGNIFICANT) 

Performance  Criteria:  2.2.  arc  seconds  for  large  scale,  12.5  for  full  disk. 

MOP  4-2-10:  resolution  of  the  detected  solar  magnetic  field.  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  5  arc  seconds  at  large  scale. 

MOP  4-2-1 1:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the  image 
enhancement  of  gray  scale  and  pseudo  color.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  4-2-12:  rate  of  optical  scan  conversion.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  4-2-13:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the  adequacy 
of  stored  images  compared  to  the  live  images.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  4-2-14:  subjective  rating  by  trained  NGDC  analysts  of  the  adequacy  of  archived  solar 
data. 

Performance  Criterion:  adequate. 

MOP  4-2-15:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the 
spectrographic  output  data.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  4-2-16:  subjective  rating  by  trained  SOON  analysts  and  programmers  of  the  digital 
images  (scaled,  standardized,  and  superimposed).  (SIGNIFICANT) 

Performance  Criterion:  adequate. 


MOP  4-2-17:  amount  of  degradation  in  current  provided  operability.  (SIGNIFICANT) 
Performance  Criterion:  no  degradation. 

MOE  4-3:  Capability  of  the  SOON's  analog  to  digital  (A/D)  converters. 

MOP  4-3-1:  perforins  required  A/D  converter  functions  performed.  (SIGNIFICANT) 
Performance  Criterion:  adequate. 

MOP  4-3-2:  number  of  SOON  significant  outages  caused  by  an  A/D  converter. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected.  A  significant  outage  is  defined  in  AFSFCR  105-1 
Attachment  4  (refer  to  Figure  3-1). 

MOP  4-3-3:  number  of  A/D  converters  failures.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected.  This  is  based  on  the  requirement  for  IX  months  of 
operation  without  a  failure. 

MOE  4-4:  Capability  of  the  SOON’s  digital  to  analog  (D/A)  converters. 

MOP  4-4-1:  performs  required  D/A  converter  functions.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  4-4-2:  number  of  SOON  significant  outages  caused  by  a  D/A  converter. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected.  A  significant  outage  is  defined  in  AFSFCR  105-1 
Attachment  4  (refer  to  Figure  3- 1 ). 

MOP  4-4-3:  number  of  D/A  converter  failures.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

COI-5:  Is  the  SEON  software  supportable? 

MOE  5-1:  Usability  of  the  SEON  software. 

MOP  5-1-1:  The  usability  of  the  SEON  software.  (CRITICAL) 

Performance  Criterion:  Subjective. 

MOP  5-1-2:  number  of  tested  software  changes  that  cannot  be  introduced  into  the 
system  without  rebooting  the  computer. 

Performance  Criterion:  zero  detected. 

MOE  5-2:  Maturity  of  the  SEON  software. 

MOP  5-2-1:  The  maturity  of  the  SEON  software.  (SIGNIFICANT) 


Performance  Criterion:  Subjective. 


MOE  5-3:  Maintainability  of  the  SEON  software. 

MOP  5-3-1:  The  maintainability  of  the  SEON  software. 

Performance  Criterion:  Subjective. 

MOE  5-4:  Support  resources  of  the  SEON  software. 

MOP  5-4-1:  The  support  resources  of  the  SEON  software.  (SIGNIFICANT) 

Performance  Criterion:  Subjective. 

MOP  5-4-2:  system  impact  of  testing  approved  software  changes. 

Performance  Criterion:  subjective. 

MOE  5-5:  Life-cycle  process  of  the  SEON  software. 

MOP  5-5-1:  The  life-cycle  process  of  the  SEON  software.  (SIGNIFICANT) 

Performance  Criterion:  Subjective. 

MOE  5-6:  Capability  of  the  programmers  to  perform  their  mission. 

MOP  5-6-1:  time  required  to  update  and  install  completely  new  source  software  distribution 
into  the  system.  (SIGNIFICANT) 

Performance  Criterion:  less  than  X  hours. 

MOP  5-6-2:  number  of  approved  software  changes  the  programmers  cannot  introduce  into 
the  application  software.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  5-6-3:  number  of  software  development  tools  the  programmers  cannot  effectively  use. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  5-6-4:  number  of  program  documentation  items  the  programmers  cannot  update. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  5-6-5:  number  of  programming  tasks  that  cannot  be  performed  due  to  inadequate 
training.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

COI-6:  Does  the  SEON  adequately  report  solar  events? 
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MOE  6-1  Capability  of  the  network  to  transmit  and  receive  messages  me  the  Automated 
Weather  Network  (AWN) 

MOP  6-1-1:  percent  of  message  formats  transmitted  LAW  Air  f  orce  Space  forecast  (  enter 
Pamphlet  (AFSFCP)  105-5.  (SIGNIFICANT) 

Performance  Criterion:  (refer  to  Table  3-1). 

MOP  6-1-2:  time  between  solar  activity  and  transmission  ot  the  required  message. 
(CRITICAL) 

Performance  Criterion:  IAW  AFSFCR  105-1,  Attachment  4  (refer  to  Figure  3-2). 

MOP  6-1-3:  number  of  errors  between  transmitted  and  received  messages.  (SIGNIFICANT 
Performance  Criterion:  zero  detected. 

MOP  6-1-4:  number  of  solar  events  the  analysts  are  unable  to  respond  to  within  time 
constraints  listed  in  Air  Force  Space  Command  Regulation  (AFSCRj  105-1.  (CRITICAL) 
Performance  Criterion:  zero  detected. 

COI-7:  Is  the  SEON  supportable? 

MOE  7-1:  Logistics  supportability,  readiness,  and  maintainability  of  the  SEON. 

MOP  7-1-1:  mean  time  between  critical  failure  (MTBCF)  of  the  RSTN.  (SIGNIFICANT) 
Performance  Criterion:  a  minimum  of  750  hours. 

MOP  7-1-2:  MTBCF  of  the  SOON.  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  750  hours. 

MOP  7-1-3:  mean  time  between  maintenance  corrective  (MTBMc).  (SIGNIFICANT) 
Performance  Criterion:  a  minimum  of  200  hours. 

MOP  7-1-4:  mean  repair  time  (MRT)  ot  the  SEON.  (SIGNIFICANT) 

Performance  Criterion  a  maximum  of  4  hours. 

MOP  7-1-5:  MRT  of  the  embedded  computer  resources.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  I  hour. 

MOP  7-1-6:  MRT  of  the  SFIR.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1.5  hours. 

MOP  7-1-7:  Max  time  to  repair  (Max  Tl  R }  for  the  SFIR  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  ot  X  hours. 


MOP  7-l-X:  Max  TTR  for  the  CCD.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1  hour. 

MOP  7-1-9:  Max  TTR  for  the  UPS.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  1  hour. 

MOP  7-1-10:  Max  TTR  for  the  S EON.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  3  hours  for  95 %  of  all  failures. 

MOP  7-1-11:  number  of  special  tools  required  to  maintain  the  SEON.  (SIGNIFICANT) 
Performance  Criterion:  zero  detected. 

MOP  7-1-12:  number  of  peculiar  test  equipment  required  to  maintain  the  SEON. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  7-1-13:  percent  of  initial  spares  support  listing  (ISSL)  items  filled  at  site  prior  to 
QOT&E. 

Performance  Criterion:  a  minimum  of  X()%. 

MOP  7-1-14:  percent  of  critical  ISSL  items  filled  at  site  prior  to  QOT&E.  (SIGNIFICANT) 
Performance  Criterion:  100%. 

MOP  7-1-15:  UTR  of  the  SEON  system. 

Performance  Criterion:  97%. 

MOE  7-2:  Maintainability  of  the  SEON. 

MOP  7-2-1:  Max  TTR  for  the  SEON.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  3  hours  for  95%  of  all  failures. 

MOP  7-2-2:  number  of  failures  not  detected  through  test  points  using  full  resources. 
(SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  7-2-3:  percent  of  detected  faults  corrected  by  replacing  one  Line  Replaceable  Unit 
(LRU).  (SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  90%. 

MOP  7-2-4:  percent  of  detected  faults  corrected  by  replacing  a  maximum  of  two  LRUs. 
(SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  95%. 


MOP  7-2-5:  percent  of  detected  faults  corrected  by  replacing  a  maximum  of  three  LRUs. 
(SIGNIFICANT) 

Performance  Criterion:  a  minimum  of  W. 

MOP  7-2-6:  weekly  preventive  maintenance  inspection  (PM I)  times  for  maintaining  the 
upgraded  portions  of  the  SEON.  (SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  30  minutes. 

MOP  7-2-7:  weekly  PMI  times  for  maintaining  the  upgraded  portions  of  the  RSTN. 
(SIGNIFICANT) 

Performance  Criterion:  a  maximum  of  30  minutes. 

MOP  7-2-8:  amount  of  increased  time  required  to  perform  weekly  PMI  for  maintaining  the 
non-upgraded  portions  of  the  RSTN.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  7-2-9:  amount  of  increased  time  required  to  perform  weekly  PMI  for  maintaining  the 
non-upgraded  portions  of  the  SOON.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  7-2-10:  number  of  failures  requiring  more  than  removal  and  replacement  of  defective 
LRUs  on  the  new  SFIR,  CCDs,  IDAS,  or  computer  resources.  (SIGNIFICANT) 

Performance  Criterion:  zero  detected. 

MOP  7-2-11:  depot-level  task  completion  time.  (SIGNIFICANT) 

Performance  Criterion:  Less  than  6  hours. 

MOP  7-2-12:  MTBCF  of  the  SFIR  and  discrete  frequency  radiometer.  There  is  no 
performance  criteria  for  this  MOP. 

MOP  7-2-13:  up  time  ratio  (UTR)  of  the  RSTN  system. 

Performance  Criterion:  a  minimum  of  97%. 

MOP  7-2-14:  UTR  of  the  SOON  system. 

Performance  Criterion:  a  minimum  of  97%. 

MOE  7-3:  Adequacy  of  the  technical  data  provided  with  the  upgraded  SEON. 

MOP  7-3-1:  number  of  significant  outages  caused  by  erroneous  technical  data.  (CRITICAL) 
Performance  Criterion:  zero  detected. 

MOP  7-3-2:  subjective  assessment  by  qualified  programmers,  technicians,  and  analysts  of  the 
technical  data.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 


MOE  7-4:  Training  provided  to  support  the  upgraded  SEON. 

MOP  7-4-1:  subjective  assessment  by  qualified  SEON  operators,  maintainers,  and  analysts  of 
the  training.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  7-4-2:  number  of  discrepancies  between  maintenance  AFSC  job  descriptions  and  the 
tasks  performed. 

Performance  Criterion:  zero  detected. 

MOP  7-4-3:  number  and  skill  level  of  personnel  required  to  perform  any  SEON  maintenance 
routine.  (SIGNIFICANT) 

Performance  Criterion:  one  30452  technician. 

MOP  7-4-4:  adequacy  of  the  course  chart  for  maintenance.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  7-4-5:  adequacy  of  plan  of  instruction  for  maintenance.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  7-4-6:  adequacy  of  the  provided  student  maintenance  training  material. 

(SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  7-4-7:  adequacy  of  the  maintenance  training  program  and  training  equipment  plan. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  7-4-X:  number  of  discrepancies  between  operations  AFSC  job  descriptions  and  the 
tasks  performed. 

Performance  Criterion:  zero  detected. 

MOP  7-4-9:  adequacy  of  the  course  chart  for  operations.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 

MOP  7-4-10:  adequacy  of  the  plan  of  instruction  for  operations.  (SIGNIFICANT) 
Performance  Criterion:  adequate. 

MOP  7-4-1  1:  adequacy  of  the  provided  student  operations  training  material. 

(SIGNIFICANT) 

Performance  Criterion:  adequate. 


MOP  7-4-12:  adequacy  of  the  operations  training  program  and  training  equipment  plan. 
(SIGNIFICANT) 

Performance  Criterion:  adequate. 

COI-8:  Does  the  SEON  facility  provide  adequate  power  and  environmental  support? 

MOE  8-1:  Capability  of  the  uninterruptable  power  source  (UPS). 

MOP  8-1-1:  time  the  UPS  provides  power  to  the  equipment  connected  to  it  in  a  generator 
backed-up  configuration.  (CRITICAL) 

Performance  Criteria:  long  enough  for  the  backup  generator  to  come  on-iine  with  no  loss  of 
SEON  data. 

MOP  8-1-2:  time  the  UPS  provides  power  to  the  equipment  connected  to  it  in  a  non¬ 
generator  backup  configuration.  (CRITICAL) 

Performance  Criterion:  long  enough  for  the  equipment  to  be  orderly  and  safely 
powered-down  with  no  loss  of  SEON  data. 

MOE  8-2:  Environmental  control  of  the  Palehua  Observatory. 

MOP  8-2-1:  range  of  temperature  in  the  SOON  equipment  room.  (SIGNIFICANT) 
Performance  Criterion:  60"  F  to  80"  F. 

MOP  8-2-2:  range  of  temperature  in  the  RSTN  equipment  room.  (SIGNIFICANT) 
Performance  Criterion:  50"  F  to  90"  F. 

MOP  8-2-3:  range  of  relative  humidity  in  the  SOON  equipment  room.  (SIGNIFICANT) 
Performance  Criterion:  30%  to  60%. 

MOP  8-2-4:  range  of  relative  humidity  in  the  RSTN  equipment  room.  (SIGNIFICANT) 
Performance  Criterion:  25%  to  75%. 

MOP  8-2-5:  the  amount  of  operational  impact  caused  by  EMI  encountered  by  the  SEON. 
(SIGNIFICANT) 

Performance  Criterion:  No  operational  impact. 

MOP  8-2-6:  the  amount  of  operational  impact  caused  by  RFI  encountered  by  the  SEON. 
(SIGNIFICANT) 

Performance  Criterion:  No  operational  impact. 


COl-9:  Is  the  human  facto  s  engineering  of  the  SEON  adequate? 

MOE  9-1:  Human  factors  engineering  (HFE)  aspects  of  the  upgraded  SEON. 

MOP  9-1-1:  number  of  significant  outages  caused  by  improper  HFE  considerations. 
(CRITICAL) 

Performance  Criterion:  zero  detected. 

MOP  9-1-2:  subjective  assessment  by  qualified  operators,  muintainers,  and  ana'ysts  of  the 
HFE  characteristics.  (SIGNIFICANT) 

Performance  Criterion:  adequate. 
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